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SUMMARY 


The  purpose  of  this  research  effort  was  to  develop  conceptual 
descriptions  for  a  variety  of  reduced  physical  fidelity  (RPF)  training 
devices  that  might  be  cost-effectively  substituted  for  actual  equipment 
trainers  (AET)  in  selected  areas  of  Army  maintenance  training.  An 
associated  task  v/as  to  develop  an  evaluation  methodology  for  determining  the 
training  effectiveness  of  the  conceptual  devices  should  they  be  developed 
and  procured  by  the  Army.  The  effort  was  focused  on  segments  of  Advanced 
Individual  Training  of  wheel  and  track  vehicle  maintenance  pei sonnel  and 
turret  maintenance  personnel. 

The  first  area  of  activity  involved  identification  of  certain  training 
requirement  areas  for  which  the  Army  might  consider  development  of  RPF 
alternatives  to  AETs.  The  second  major  area  of  activity  was  the  cevelopment 
of  conceptual  descriptions  of  RPF  alternatives  for  specific  skill/task 
segments  of  maintenance  training,  while  the  third  treated  various 
considerations  related  to  evaluation  of  the  training  effectiveness  of  any 
resultant  RPF  devices. 

These  activities  resulted  in  identification  of  four  segments  of  Army 
maintenance  training  in  which  cost-effective  RPF/AET  substituticn  might  be 
made  and  conceptual  descriptions  of  eight  RPF  devices  to  be  cons idered  for 
that  substitution.  Two  devices  are  recommended  for  use  in  teaching 
troubleshooting  of  engines  and  related  systems;  one  device  is  identified  for 
teaching  troubleshooting  of  track/suspension  systems;  one  device  is 
described  for  teaching  removal/replacement  of  power  packs;  and  our  device 
concepts  are  presented  for  teaching  troubleshooting  of  turret  ele  :trical  and 
hydraulic  systems. 

A  general  methodology  is  then  presented  for  df  termining  tl e  training 
effectiveness  of  any  of  the  RPF  devices  which  might  be  produced.  This 
methodology  considers  such  aspects  as  the  device's  effectivi ness  in  a 
variety  of  training  settings,  its  performance  characteristic!,  and  its 
measurement  and  feedback  capabilities.  Also  described  are  the  procedural 
steps  for  the  evaluation  effort  and  the  types  of  expertise  needed  on  the 
team  conducting  the  evaluation. 
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] .  BACKGROUND 


INTRODUCTION 

Ttie  training  of  military  maintenance  personnel  has  recCived  increasing 
attention  in  recent  years.  Four  factors  appear  to  be  primarily  responsible 
for  this  concern: 

1.  Modern  weapons  systems  are  becoming  increasingly  complex  end 
sophisticated  with  resultant  requirements  for  more  highly 
skilled  maintenance  personnel, 

2.  Costs  of  training  maintenance  personnel  have  increased. 

3.  Costs  of  training  aids  and  devices  used  in  maintenance 
training  are  correspondingly  greater  as  weapons  systems 
increase  in  complexity  and  sophistication. 

A.  The  relative  effectiveness  of  training  aids  and  devices 
currently  used  in  maintenance  training  is  uncertain. 

Added  to  these  four  factors  is  a  fifth  overriding  concern,  the  importance  of 
maintenance  to  the  operational  capabilities  of  military  units.  In  the  total 
context,  it  is  easy  to  understand  this  increasing  attention  upon  maintenance 
training  and  the  resultant  efforts  to  reduce  its  cost,  without 
sacrificing  its  quality  and  effectiveness,  and  to  increase  its  effectiveness 
in  skill  shortfall  areas. 

There  have  been  numerous  efforts  aimed  at  alleviating  the  difficulties 
underlying  these  concernc.  For  example,  there  have  been  attempts  to  produce 
hardware  that  requires  less  maintenance  or  can  be  maintained  by  less  skilled 
personnel,  but,  while  significant  progress  has  been  made  in  some  instances, 
the  trend  toward  higher  skill  requirements  has  continued.  Concerted  efforts 
have  also  been  made  to  improve  the  training  process  itself.  Perhaps  the 
most  significant  of  such  efforts  has  been  the  development  of  the 
Instructional  Systems  Development  (ISD)  process  i./ ,  This  systematic 
procedure  begins  with  analysis  of  the  actual  tasks  performed  by  military 
personnel  on  the  job  and  identification  of  the  tasks  considered  most 
critical  to  job  performance.  Institutional  training  is  then  focused  on  the 
more  critical  tasks,  with  the  less  critic.  1  tasks  typically  beii g  assigned 
to  on-the-job  training  (OJT).  Elimination  of  noncritical  material  from 
institutional  training  usually  results  in  shorter,  less  costly,  p  oficiency- 
based  courses,  who^e  graduates  are  presumed  to  be  more  highly  skilled  in  the 
elements  considered  crucial  to  optimum  performance  on  the  jnb  The  TSD 
process,  therefore,  can  have  a  direct  and  significant  impact  oi  the  first 
two  of  the  four  concerns  for  military  maintenance  training  lifted  above. 


~  Interservice  Procedures  for  Instructional  Systems  Di  velopment . 
TRADOC  PAM  350-30,  Headquarters,  U.S.  Army  Training  and  Doctrire  Command, 
Fort  Monroe,  Va.,  August  1975. 
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The  latter  two  areas  of  concern  regarding  the  cost  and  effectiveness  of 
training  aids  and  devices,  particularly  actual  equipment  used  as  training 
devices,  are  continuing  proMt-ms  and  are  the  focus  of  the  research  effort 
described  in  this  report. 

MILITARY  PROBLEM 

The  Array,  like  all  the  services,  has  become  increasingly  concerned  with 
its  raaintenance  training.  Maintenance  problems  are  compounded  by  the  wide 
variety  of  equipment  and  vehicles  peculiar  to  combat  units  and  by  the 
unfavorable  field  environment  in  which  much  Army  maintenance  necessarily 
takes  place.  Further,  the  new  generations  of  Army  weapons  systems  (e.g.. , 
the  XM-1  tank)  are  costly  and  complex. 

The  teaching  of  maintenance  tasks  in  the  Army  has  traditionally 
been  organized  around  a  theory-first,  practice-later  paradigm.  Also,  the 
Army  has  generally  followed  the  traditional  practice  of  placing  major 
reliance  on  the  use  of  Actual  Equipment  Trainers  (AET)  1.1  as  the  principal 
vehicles  for  "hands-on"  practice  in  training.  Training  has  typically 
consisted  of  the  presentation  of  important  background  knowledge  through 
classroom  lecture,  followed  by  demonstration  of  the  maintenance  task  by  the 
instructor.  Small  groups  of  students  then  practice  the  task  on  an  AET,  and 
all  members  of  the  class  cover  the  same  material  at  the  same  rate.  There 
are  two  significant  disadvantages  to  this  mode  of  instruction:  (1)  the 

instruction  is  generally  aimed  at  the  middle  ability  level  of  the  class, 
thus  frustrating  both  the  higher-ability  and  lower-ability  students;  and  (2) 
because  they  work  in  groups  and  all  groups  move  at  the  same  pace,  not  all 
students  are  able  to  practice  all  tasks  on  the  equipment.  In  some  cases, 
there  is  also  reliance  on  paper-and-penc  i  1  tests  to  measure  student 
proficiency.  These  tests  measure  the  basic  knowledge  taught  in  the 
classroom  reasonably  well,  but  are  weak  as  means  of  evaluating  those 
porc’ors  of  maintenance  training  that  focus  on  phvsical  manipulation  of 
tools  and  equipment. 

Consideration  of  these  problems  in  light  of  ejctensive  research  into 
human  learning  and  advancements  in  educational  technology  that  have  occurred 
over  the  last  decade  or  two  has  brought  about  a  fundamental  change  in 
military  maintenance  training  philosophy.  There  has  been  a  shift  away  from 
the  lock-step  approach  in  training  toward  an  approach  that  emphasizes 
individualized  instruction  and  employs  such  procedures  as  self-pacing 
proficiency-based  training,  performance  mastery,  and  the  like.—' 


-The  term  AET  refers  to  items  of  actual  equii'ment  that  are  used  as 
training  devices.  AETs  may  be  entire,  end-item  systems  such  as  trucks,  or 
they  may  be  end-item  subsystems  or  components  such  as  truck  engines.  AETs 
may  be  whole,  i.e.,  in  operational  form,  or  they  may  be  modified  for  use  as 
training  devices  (e.g.,  truncated,  cut-away,  and/or  mounted  on  a  platform). 

2/ 

Taylor,  J.  Establishing  the  Concepts _ and  Techniques  of  Performance 

Oriented  Training  in  Army  Training  Centers:  A  Summary  Report.  Technical 
Report  75-21,  Human  Resources  Research  Organization,  Alexandria,  Va.,  June 
1975. 
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As  would  be  the  case  in  any  sizable  educatio  lal/traLning  system,  this 
change  from  lock-step  to  self-paced J,/  instruction  las  not  come  about  without 

the  occurrence  of  major  problems.  Significant  imong  these  are  problems 

related  to  the  AETs  currently  being  used  in  mainte  lance  training.  In  self- 
paced  instruction,  the  individual  student  is  required  to  demonstrate 
proficiency  on  each  required  taek  before  advancinj  to  the  next  task  in  the 
instructional  sequence.  The  shift  from  lock-step  to  self-paced  maintenance 
training,  therefore,  is  placing  greater  and  greater  demands  on  AETs 

currently  supporting  the  training.  Many  AETs  cannot  hold  up  under  the 

increased  usage  and  deteriorate  rapidly.  In  addition,  the  task-or ientec 
nature  of  self-pacec  training  is  creating  a  need  f ir  even  more  equipment  to 
support  this  newer  method  of  training. 

As  has  been  noted,  AETs  have  traditionally  been  the  mainstay  among 
maintenance  training  devices,  and  their  use  has  be -n  predicated  on  a  number 
of  presumed  advantages: 

•  they  provide  a  high  degree  of  face  validit/,  or  realism,  to  the 
s  tudent ; 

•  they  are  usually  readily  available; 

•  they  do  not  require  large  developmental  co  ts  or  lengthy  develop¬ 
mental  cycles  such  as  are  sometimes  associated  with  synthetic 
training  devices;  and 

•  items  of  equipment  which  have  become  unserviceable  due  to  damage 
or  age  can  be  removed  from  the  operating  inventory  and  used  as 
maintenance  training  devices  at  very  low  cost. 

The  ‘^irst  of  these  presumed  advantages,  provision  of  realism  to  the  student, 
is  considered  by  maintenance  training  personnel  to  be  the  most  important. 
It  is  virtually  a  basic  "article  of  faith"  among  most  maintenance  training 
personnel  that  the  only  effective  way  to  teach  most  "hands-on"  skills  is 
through  the  use  of  actual  equipment. 

There  are,  however,  a  number  of  disadvantages  to  relying  upon  AETs  as 
tr.iining  devices: 

•  in  the  case  of  a  new  system  (e.g.,  the  XM-1  tank),  there  are  no 
unserviceable  items  in  the  inventory,  and  expensive  operational 
equipment  must  be  tied  up  in  supporting  training  functions  instead 
of  supporting  the  functions  for  which  it  was  designed; 

•  AETs  typically  have  few,  if  any,  features  included  specifically  to 
enhance  the  training  process; 


—  ks  used  here,  the  term  "self-paced"  refers  to  an  instructional 
program  in  which  the  student's  day-to-day  instructional  events  and  pace  at 
which  he  moves  through  the  program  are  dependent  upon  the  performance  he 
exhibits.  The  term  is  not  intended  hero  to  connote  that  necessarily  the 
student  chooses  the  pace  as  he  pleases,  though  it  is  not  incompatible  with 
such  choice. 


•  many  AETa  are  carried  on  the  inventory  as  training  aids  and  do  not 
receive  the  engineering  changes  that  operational  equipment  receives 
through  the  Department  of  Army  modification  work  order  or  product 
improvement  program,  thereby  tending  to  become  obsolescent  and  out 
of  date; 

•  AETs  usually  provide  little  or  no  trainee  response-measuring 
capability  or  means  of  providing  feedback  to  the  trainee; 

•  AETs  have  limited  capabilities  for  malfunction  insertion  and/or 
demonstration  of  non-normal  operation; 

•  AETs  normally  receive  a  low  priority  for  repair  parts  and  con¬ 
sequently,  may  be  inoperable  pending  receipt  of  repair  parts;  and 

•  AETs  often  are  limited  to  nondestructive  uses  of  tht'  equipment, 
thus  restricting  their  usefulness  in  many  training  applications. 

As  can  be  inferred  from  these  disadvantages,  there  h"ve  l^een  serious 
questions  raised  about  the  use  of  AETs  in  terms  of  both  cost  and 
effectiveness  considerations.  For  example,  a  turret  for  the  XM-1  tank  is  an 
expensive  item,  and  when  modified  for  use  as  an  AET  its  cost  wiuld  be  even 
greater.  In  spite  of  this  cost,  its  utility  in  terms  of  training 
effectiveness  and  efficiency  would  still  be  limited.  Further,  the  AET  is 
often  incompatible  with  the  individualized,  self-paced  instructional 
approach  toward  which  the  Army  is  moving  in  its  maintenance  training. 

Recent  advances  in  training  technology  have  produced  a  wide  array  of 
newei  instructional  media,  media  that  aie  geneidlly  more  compatible  with  the 
training  methods  now  espoused  by  the  Army.  Included  in  their  number  are 
devices  of  reduced  physical  fidelity  (RPF).  Many  of  th ’se  devices 

incorporate  such  training-related  features  as  automated  performance 
measurement  and  immediate  feedback  to  trainees,  capabilities  which  AETs  do 
not  possess. 

As  a  result  of  these  factt)rs  and  the  Army's  general  concern  for 
improving  its  maintenance  training  better  to  meet  operational  needs,  the 
Army  finds  it  must  seek  alternatives  to  AETs  in  many  areas.  This  is  not 
meant  to  suggest  that  actual  equipment  has  no  place  in  Army  maintenance 
training,  but  th'.t  media  alternatives  (including  AETs)  must  be  examined  in 
terms  of  their  overall  cost  effectiveness  in  meeting  maintenance  training 
requirements  as  part  of  a  total  system  approach.  Thus,  where  AETs  are 
uniquely  suited  to  the  training  requirements,  where  they  possess  cost- 
effectiveness  advantages,  or  where  no  alternative  exists,  they  will  continue 


—  RPF  devices  can  be  either  two-  or  three-dimensional  representations 
of  actual  equipment  or  systems  within  the  equipment.  T'ese  devices  include 
a  variety  of  trainers  that  usually  display  the  system  under  study  on  a  panel 
and  are  either  hardwired  or  computer-controlled.  Many  are  factory- 
programmed,  but  others  may  be  programmed  by  the  user.  Drawings  are  normally 
used  to  show  components,  but  some  actual  components  may  be  used  to  attain  a 
certain  degree  of  realism. 
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to  be  the  instructional  medium  of  choice.  —  However,  in  those  areas  where 
alternatives  exist,  or  can  be  developed,  and  have  cost  and  training  advan¬ 
tages  over  the  AET,  it  is  prudent  f-hat  the  Army  pursue  such  alternatives. 
Thus,  the  general  military  problem  addressed  in  the  research  effort  reported 
here  is  the  role  of  AETs  in  Army  maintenance  training  and  the  identification 
of  alternatives  to  AETs  that  have  promise  of  cost  and/or  training  benefit. 

RESEARCH  PROBLEM 

From  the  research  point  of  view,  the  military  problem  described  has 
several  facets.  First,  what  are  the  types  of  maintenance  skills  and 
knowledges  for  which  training  devices  a.  2  or  may  be  used?  Tliis  requires 
behavioral/instructional  analysis.  Next,  what  are  the  different  types  of 
devices?  The  AET  is  the  major  device  type  used  in  maintenance  training,  but 
developments  in  training  technology  and  simulation  have  led  to  development 
of  a  variety  of  RPF  devices  that  are  or  can  be  used  in  maintenance  training. 
Finally,  how  can  the  effectiveness  of  a  given  devicv  in  teaching  specific 
maintenance  skills  and  knowledges,  or  classes  of  such  skills  and  knowledges, 
be  evaluated? 

The  present  research,  therefore,  has  addressed  three  areas  as  they 
relate  to  Army  maintenance  training:  (1)  skill/knowledge  definition;  (2) 
device  alternatives;  and  (3)  device  evaluation.  The  effort  is  part  of  the 
overall  training  device  research  program  of  the  U.S.  Army  Project  Manager 
for  Training  Devices  (PM  TRADE),  The  basic  thrust  of  PM  TRADE'S  efforts  is 
the  development  of  more  effective  devices  for  the  Army,  devices  that  allow 
achievement  of  necessary  training  goals  at  minimum  cost.  The  research  aim 
has  been  the  identification  and  conceptual  description  of  potentially  cost- 
effective  alternatives  to  AET  devices  for  use  in  selected  areas  of  Army 
maintenance  training  and  development  of  a  methodology  for  evaluating  such 
ilternative  devices. 

ORGANIZATION  OF  REPORT 

The  main  thrust  of  the  effort  reported  here  has  been  the  development  of 
conceptual  descriptions  of  RPF  devices.  Therefore,  the  emphasis  in  the  main 
body  of  the  report  will  be  o,.  these  RPF  concepts  and  the  general  areas  of 
training  to  which  they  are  addressed.  The  bulk  of  the  detail  relating  to 
the  various  tasks  or  activities  that  underlie  the  RPF  concepts  is  relegated 
to  the  various  appendixes. 

The  report  is  comprised  of  four  sections,  including  t’e  present  intro¬ 
duction.  In  the  second  section  the  general  approach  folloved  in  the  overall 
effort  is  described  in  terms  of  the  specific  research  tasks  that  were 
involved.  However,  much  of  the  detailing  of  these  tasks  is  covered  in  the 
appendixes.  Section  three  describes  the  various  areas  of  training  of 
principal  concern  and  presents  a  description  of  the  various  RPF  concepts 
developed.  It  also  includes  a  discussion  of  a  device  training  effectiveness 
evaluation  methodology  as  developed  in  the  present  effort.  Finally,  the 


—  AETs  may  be  used  in  conjunction  with  RPF  devices  to  facilitate  the 
transition  to  actual  equipment. 


fourth  section  presents  a  discussion  of  Army  maintenance  training  and  AET 
and  RPF  device  requirements.  The  appendixes  to  the  report  are  referenced  as 
appropriate  in  the  main  text. 


II.  APPROACH 


GENERAL  SCOPE 

The  basic  goals  of  this  effort  have  been  to  Identify  areas  in  which  RPF 
substitution  for  AET  devices  might  be  considered,  to  specify  the  conceptual 
form  for  such  RPF  devices,  and  to  provide  proceduial  guidance  for  the  future 
training  effectiveness  evaluation  of  such  RPF  devices  should  they  be 
developed  to  the  point  of  prototype  fabrication-  In  pursuit  of  these  goals, 
a  systematic  procedure  has  been  followed  in  which  current  Army  maintenance 
training  programs  were  examined  and  certain  device  requirement  areas 
identified.  The  approach  has  involved  a  combination  of  survey  analysis, 
rational  analysis,  and  professional  judgment  based  on  empirical  data 
wherevet  possible. 

The  first  major  research  activity  in  the  effjrt  required  identification 
of  training  requirement  areas  for  which  the  Army  might  reasonably  consider 
development  of  RPF  alternatives  to  AETs.  The  second  maj<3r  area  of  activity 
was  the  development  of  conceptual  descriptions  of  such  RPF  alternatives, 
while  the  third  treated  various  considerations  pertinent  to  evaluation  of 
any  resultant  RPF  devices  with  reference  to  training  effectiveness. 

PROJECT  TASKS 

The  accomplishment  of  the  objectives  of  the  three  areas  of  project 
activity  jujt  described  required  that  six  specific  research  tasks  be 
executed.  The  first  two  tasks  concerned  the  first  area,  i.e., 

identification  of  the  maintenance  training  areas  of  primary  concern  to  the 
Army  in  the  present  context.  The  second  area  (development  of  RPF  conceptual 
descriptions)  also  involved  two  specific  tasks,  while  the  third  area 
involved  a  single  task.  In  addition,  a  sixth  research  task  concerned  with 
reporting  was  carried  out,  but  it  dealt  with  all  areas  of  project  activity. 

Throughout  the  effort,  guidance  was  provided  to  tie  Seville  project 
team  by  an  Army  Study  Advisory  Group  (SAG).  The  SAG  consisted  of 
representatives  from  the  Army  Training  and  Doctrine  Command  (TRADOC),  as 
well  as  representatives  of  PM  TRADE.  Through  a  series  of  meetings  and 
interim  reports  by  the  Seville  team,  the  SAG  was  kept  informed  of  project 
activities  and  plans  and  was  able  to  provide  technical  guidance  to  the 
research  team  on  a  regular  basis.  Prior  Co  initiation  of  the  first  task, 
Seville  personnel  were  required  to  present  to  the  SAG  a  detailed  plan  for 
execution  of  the  six  tasks,  and,  in  fact,  certain  revisions  to  the  plan 
were  made  as  a  result  of  this  interaction.  It  should  also  be  noted  that 
the  SAG  played  an  active  role  in  making  some  of  the  arrangements  for 
training  site  visits  made  by  the  research  team. 

As  noted,  six  specific  tasks  were  formulat  id  for  achievement  of  the 
overall  research  objectives.  The  six  tasks  were  as  follows: 

1.  Investigation  of  Use  of  AETs  in  Maintena  ice  Trailing 

2.  Identification  of  Potential  AET  Replacement  Areas 

3.  Evaluation  of  Existing  RPF  Alternatives 


4.  Development  of  New  RPF  Concepts 

5.  Development  of  Training  Effectiveness  Evaluation  Requirements 

6.  Reporting 

General  descriptions  of  the  six  tasks  follow,  and  additional  details  are 
given  in  the  various  appendixes. 

Task  1  -  Investigation  of  Use  of  AETs  in  Maintenance  Training 

The  general  objective  of  Task  1  was  to  determine  the  nature  of  Army 
maintenance  training,  its  objectives  and  organization,  and  the  actual  uses 
being  made  of  AETs  and  other  devices  in  that  training.  The  principal 
research  activity  involved  was  a  survey  of  the  use  of  AET  devices  in  the 
institutional  training  of  Army  personnel  to  maintain  track  and  wheel 
vehicles,  including  maintenance  of  turrets.  Advanced  Individual  Training 
(AIT)  courses  for  both  organizational  and  DS/GS  level  mechanics  were 
included  (see  Appendix  A).  Some  twelve  different  hOS-produclng  maintenance 
courses  at  three  major  Army  training  centers  were  surveyed.  The  survey  was 
designed  to  determine  the  nature  of  skills  taught,  the  general  organization 
of  the  courses  of  instruction,  the  AETs  and  other  devices  used  to  support 
instruction,  and  a  variety  of  other  relevant  fictors  such  as  training 
schedules,  student  input/output  flow,  equipment  costs,  and  the  like. 

Survey  of  the  practices  of  non-Army  counterpart  maintenance  training 
programs  was  also  conducted  in  Task  1.  This  surve>  touch* d  on  the  programs 
of  the  other  military  services,  as  well  as  selected  nonmi]itary  agencies. 

Further  details  of  the  surveys,  including  survey  procedures,  Interview 
guides,  and  sites  and  persons  surveyed,  are  given  in  Apper dlxes  A  and  B. 

Task  2  -  Identification  of  Potential  AET  Replacement  Areas 


The  objective  of  Task  2  was  to  identify  those  areas  of  AET  usage  in 
Arny  vehicle  maintenance  training  for  which  an  RPF  alternative  might  be  cost 
effectively  substituted.  Since  Tasks  3  and  4  required  the  identification  or 
development  of  RPF  alternatives  to  specific  AET  devices  or  classes  of 
devices,  it  was  necessary  that  such  efforts  be  reduced  to  a  size  that  was 
manageable  and  feasible  within  the  resources  avalla! le  to  the  present  study. 
This  required  development  of  a  schema  for  establishing  an  ordering  of 
priority  for  the  investigation  and  development  of  potential  RPF  devices  to 
be  used  in  lieu  of  current  AET  devices.  The  training  survey  data  from  Task 
1  provided  the  input  information  for  Task  2. 

In  order  to  establish  the  necessary  priority  listing,  two  things  were 
done.  First,  the  various  training  courses  surveyed  were  examined  to  iden¬ 
tity  major  or  common  content  areas  of  concern.  This  resulted  in  iden¬ 
tification  of  some  24  major  training  areas.  Typical  examples  of  such  areas 
Include:  motor  tasks  in  power  plant  maintenance  by  organizational 
mechanics;  cognitive  tasks  in  power  train  maintenance  by  DS/GS  mechanics; 
and  troubleshooting  tasks  in  turret  maintenance  by  organizational  mechanics. 

The  second  activity  in  this  task  was  to  develop  and  apply  a  rating 
schema  to  the  24  training  areas  in  order  to  establisl  priorities  for 
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possible  AET  replaceicent .  The  concern  here  was  to  identify  those  areas  of 
greatest  training  need  or  greatest  potential  cost  benefit  for  the  Army.  In 
this  regard,  six  dimensions  were  identified  as  of  primary  concern: 

•  Skill  complexity 

•  Subject-matter  difficulty 

•  Commonality  to  other  courses 

•  Trainee  volume 

•  AET  instructional  utility 

•  AET  cost 

The  research  team,  based  on  the  Task  1  data  and  experience,  rated  the 
programs  on  the  above  dimensions  and  recommended  to  the  SAG  four  principal 
training  areas  for  concentration  of  effort  in  developing  RPF  alternatives 
during  Tasks  3  and  4.  Definitions  of  the  six  scale  dimensions  and  the 
rating  procedure  are  contained  in  Appendix  C,  along  with  results  of  the 
rating  procedure. 

Task  3  -  Evaluation  of  Existing  RPF  Alternatives 

On  the  basis  of  the  survey  data  (Task  1)  and  the  high-interest  training 
areas  identified  (Task  2),  existing  RPF  alternatives  to  AETs  were  examined 
for  possible  development  by  the  Army.  These  alternatives  were  examined  in 
terms  of  their  cost  effectiveness  as  possible  substitutes  for  AET  devices 
found  to  be  currently  in  use  in  Army  training. 

To  supplement  the  survey  data  developed  in  Task  1,  two  additional 
sources  were  surveyed  during  Task  3.  First,  some  75  commercial  training 
equipment  manufacturers  were  contacted  concerning  possible  RPF  devices  they 

might  know  of  or  which  they  might  be  producing.  Second,  various  military 

resear> h  agencies  conducting  research  on  training  devices  were  contacted 
concerning  their  research  on  or  knowledge  of  existing  RPF  devices.  These 
agencies  included  the  Army  Research  Institute,  the  Naval  Training  Equipment 
Center,  and  the  Air  Force  Human  Resources  Laboratory. 

Task  4  -  Development  of  New  RPF  Concepts 

The  objective  of  Task  4  was  the  development  of  new  R.’F  conceptual 
approaches  for  the  training  areas  identified  in  Task  2.  While  the 

approaches  were  not  to  be  constrained  by  current  practices,  it  was  desired 
that  the  concepts  to  be  described  be  feasible  of  accomplishment  with  a 
reasonably  high  probability  of  success.  Thus,  emphasis  was  on  concepts 

involving  extensions  of  existing  technologies,  or  new  combinations  thereof, 
rather  than  on  the  development  of  completely  new  technological  approaches. 

In  developing  these  new  conceptual  approaches,  particular  attention  was 
paid  to  training  areas  for  which  no  existing  RPFs  were  found  in  Task  3  and 
those  in  which  AET  costs  were  high  and/or  AET  training  ef fe'-.tiveness  was 
low. 
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Task  5  -  Development  of  Training  Effectiveness  Evaluation  Requirements 


The  focus  of  this  task  was  the  development  of  a  procedure  or 
methodology  to  provide  guidance  for  the  training  effectiveness  evaluation  of 
any  RFF  devices  that  might  eventually  result  from  Tasks  3  and  4.  Inputs 
into  the  development  of  the  evaluation  procedure  derived  from  several 
source  1.  The  principal  sources  were  the  general  technologies  dealing  with 
evalua;ion  design  and  with  performance  measurement.  Anoth€:r  major  source 
was  the  information  developed  in  Task  1  concerning  the  nature  of  Army 
maintenance  training  in  the  various  areas  of  concern.  An  additional  input 
was  based  on  general  experience  of  the  Seville  project  team  and  other 
research/evaiuation  personnel  familiar  with  the  evaluation  of  training 
devices. 

Task  6  -  Reporting 

The  conduct  of  this  task  involved  a  series  of  oral  reports  to  the  SAG, 
detailing  activities  on  the  various  tasks  in  the  overall  effort,  and  the 
present  report,  which  is  the  final  activity  of  this  task.  The  principal 
objectives  of  this  report  are  to  document  the  overall  activities  of  the 
effort,  to  describe  the  RPF  alternatives  that  can  be  considered  for  future 
development  by  the  Army,  and  to  set  forth  a  procedure  for  their  evaluation. 

The  frequent  interactions  between  the  research  team  and  the  SAG  vere 
essential  not  only  for  general  information  flow,  but  they  provided  a  series 
of  decision  mechanisms  with  reference  to  the  specifics  of  implementing  the 
general  research  plan  for  the  effort. 

APPROACH  SUMMARY 

The  six  tasks  described  were  aimed  at  satisfying  t^e  three  general 
research  objectives  of  identifying  the  high-interest  or  high-value  training 
areas  for  possible  RPF  development,  describing  RPF  devices  appropriate  for 
those  areas,  and  developing  a  procedural  guide  for  the  training 
effectiveness  evaluation  of  any  such  RPF  devices  tha*"  might  be  developed  by 
the  Army. 


III.  RPF  ALTERNATIVES  AND  EVALUATION 


This  section  of  the  report,  organized  into  four  general  parts,  details 
the  'various  RPF  alternatives  developed  in  Tasks  3  and  4  and  presents  an 
overview  of  the  training  effectiveness  evaluation  procedures  developed  in 
Task  5.  The  first  part  describes  the  four  Army  maintenance  training  areas 
that  were  identified  in  Task  2  as  being  the  high-interest  areas  with 
reference  to  possible  device  alternatives  to  AETs.  The  second  and  third 
parts  describe  the  RPF  device  concepts,  both  the  currently  existing  RPFs  and 
the  new  conceptual  approaches.  The  final  part  of  this  section  then  treats 
the  concept  of  device  training  effectiveness  evaluation. 

HIGH-INTEREST  TRAINING  AREAS 

As  previously  noted,  in  Task  2  the  24  different  major  training  areas  or 
domains  were  rated  on  certain  dimensions  in  order  to  id€ntify  the  training 
areas  of  greatest  interest  to  the  Army  with  referenca  to  possible  AET 
replacement.  The  four  training  content  areas  recommended  to  the  SAG  for 
further  consideration  (and  subsequently  approved  by  the  KAG)  were: 

1.  Troubleshooting  Engines  and  Related  Systems  at  the  Organizational 
Level 

2.  Troubleshooting  Track  Vehicle  Track/Suspension  Systems  at  the 
Organizational  Level 

3.  Removal/Replacement  of  Engines  and  Power  Packs  at  the  DS/GS 
Level 

4.  Troubleshooting  Turret  Electrical  and  Hydraulic  Systems  at 
the  Organizational  and  DS/GS  Levels 

The  first  three  of  these  areas  are  the  responsibility  of  wheel  and  track 

vehicle  mechanics,  while  area  four  is  the  responsibility  of  the  turret 

mechanic.  In  order  to  provide  some  background  for  the  RPF  device 
descriptions,  each  of  these  training  areas  will  be  discussed  in  terms  of  the 
general  maintenance  tasks  the  mechanic  must  perform  and  the  general  manner 
in  which  training  for  those  tasks  is  given.  The  distinction  between 

organizational  and  DS/GS  maintenance  levels  is  of  some  importance  to  the 
discussion.  These  levels  are  discussed  and  defined  in  Appendix  A. 

1 .  Troubleshooting  Engines  and  Related  Systems  at  the  Oiganizational  Level 

The  troubleshooting  activities  of  organizational  wheel  and  track 
vehicle  mechanics  primarily  involve  determination  of  a  vehicle's  operational 
readiness  and  serviceability.  In  regard  to  a  vehicle's  engine,  this 

involves  performing  a  number  of  preoperational  checks  cf  the  engine,  its 
components,  and  related  systems.  The  mechanic  assures  the  engine  starts 
quickly,  its  components  are  installed  and  operating  properly,  belts  are  at 
proper  tension,  fluids  and  lubricants  are  at  correct  levels,  and  performs  a 
number  of  other  rather  superficial  checks.  If  trouble  symptoms  appear,  the 
organizational  mechanic  is  required  to  identify  the  malfunction  that  caused 
the  83nnptom  to  appear  and  to  repair  it  if  it  is  within  his  authorization  to 
do  so.  As  an  organizational  mechanic,  his  authorization  is  only  to  the 
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component  or  subassembly  level,  meaning  he  can  remove  and  replace  a  starter, 
for  example,  but  is  not  authorized  to  repair  the  internal  parts  of  the 
starter.  That  function  is  served  by  the  DS/GS  repairman. 

The  organizational  mechanic  is  confronted  with  a  vast  array  of  wheeled 
and  tracked  vehicles  encompassing  a  wide  variety  of  gasoline,  diesel,  and 
multifuc'l  engines.  In  addition,  each  type  of  engine  is  supported  by  i:8  own 
unique  charging,  cranking,  ignition,  and  fuel  systems,  thereby  crea  ing  a 
formidable  array  of  tasks  for  the  organizational  mechanic.  To  support  its 
large  inventory  of  vehicles,  the  Army  is  required  to  train  ;reat  numbers  of 
organizational  mechanics.  The  four  organizational  maintenance  courses 
observed  in  this  study  (62B10,  62B20,  63B10,  63C10)  graduate  approximately 
10,000  trainees  annually,  with  60%  of  that  total  coming  from  the  63C10 
(Track  Vehicle  Mechanic)  course. 

Because  of  the  wide  array  of  engines  and  related  systems  to  which  the 
organizational  mechanic  is  exposed,  approximately  one-third  of  the  training 
time  in  the  four  courses  is  devoted  to  the  trainee  being  exposed  to  a  great 
deal  of  theory  regarding  the  function  and  operation  of  engines  and  related 
systems.  The  trainee  also  practices  a  number  of  physical  skills  requiring 
manipulation  of  tools  and  equipment,  and  learns  to  operate  several  types  of 
diagnostic  and  testing  devices,  including  the  tach/dwell  meter,  low  voltage 
circuit  tester  (LVCT),  and  multimeter. 

The  instruction  for  this  segment  of  training  begins  with  an 
introductory  lecture  by  one  or  more  instructors.  Trainees  usually  attend 
this  introduction  in  groups,  since  even  in  a  self-paced  coi.rse  the  trainees 
tend  to  work  together  in  groups.  In  lock-step  instructioi  ,  of  course,  the 
entire  class  is  present.  After  the  instructor  explains  the  objectives, 
instructional  procedures,  and  grading  methods  for  the  segr ent  of  training, 
the  trainees  move  to  the  equipment  to  perform  the  tasks  specified  in  the 
lesson  guide.  The  instructors  move  about  assisting  tho  trainees  where 
needed . 

Troubleshooting  of  gasoline  engines  and  related  systems  is  usually 
taught  using  Jeep  and  other  small  truck  engines  as  AETs,  most  of  them  still 
being  mounted  in  the  vehicles.  Troubleshooting  of  diesel  engines  and 
systems  is  usually  taught  using  actual  engines  mounted  on  stands.  Engines 
from  a  number  of  manufacturers  (primarily  Cummins,  Caterpillar,  and 
Continental)  are  used  in  this  training  to  expose  the  craineB  to  the  variety 
of  engines  he  might  be  required  to  maintain  in  his  unit  issignment.  The 
engines  mounted  on  stands,  of  course,  require  elaborate  systems  for 
provision  of  fuel  and  for  removal  of  exhaust  emissions.  Since  as  many  as  20 
engines  might  be  located  in  one  shop,  it  is  also  a  high-noise  area,  and 
trainees  and  instructors  are  required  to  wear  protective  devices  over  their 
ears. 


Before  an  engine  is  started,  the  trainee  make;  a  number  of 
preoperational  checks.  These  checks  vary  with  the  type  of  engine  (gasoline, 
diesel,  or  multifuel),  but  generally  include:  belt  tensions  and  condition; 
hose  condition;  fluid  and  lubricant  levels,  including  battery  and  radiator; 
and  the  condition  and  general  serviceability  of  the  carburetor,  fuel  pump, 
throttle  linkage,  distributor,  ignition  wires,  spark  plugs,  and  exhaust 
system.  Once  the  engine  is  running,  a  number  of  more  complex  tests  can  be 
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performet  ,  including:  tests  of  various  vacuum  systems;;  checking  of  the 
timing:  :hecking  engine  idle  speed  and  point  dwell  using  the  tach/dwell 
meter;  apd  various  checks  of  the  generator  and  voltage  regulator  using  the 
LVCT  and  multimeter. 

The  training  for  these  tasks  involves  the  trainee  performing  the  tasks, 
with  assistance  from  the  instructor  if  needed,  using  the  proper  tools,  test 
equipment,  and  maintenance  procedures  as  specified  in  the  appropriate 
technical  manual(s)  (TM).  The  engines  are  usually  in  normal  operating 
condition  during  this  phase  of  training,  and  the  trainee  is  therefore 
generally  exposed  to  only  normal  conditions  and  test  readings.  As  the 
trainee  performs  each  task,  he  receives  a  "go"  or  a  "no-go"  from  the 
instructor.  A  "no-go"  usually  requires  the  trainee  to  repeat  the  task  or  a 
certain  portion  of  the  task  until  the  instructor  is  satisfied  with  his 
performiince . 

Upon  completion  of  each  set  of  tasks  (i.e.,  upon  receiving  a  "go"  on 
all,  or  a  certain  peicentage  of,  the  tasks  from  the  instructor),  the  trainee 
is  subjected  to  a  performance  examination.  Ir  this  examination,  he  is 
directed  to  an  actual  engine  which  the  instruc  tor  has  "bugged"  through 
insertion  of  a  number  of  malfunctions.  A  certain  time  limit  is  established, 
and  the  trainee  must  locate  and  repair  all  the  malfunctions,  thus  returning 
the  engine  to  normal  operating  condition.  Howevi  r,  the  number  and  variety 
of  malfunctions  to  which  the  trainee  is  e  .posed  are  limited  both  by  time  and 
the  difficulty  of  "bugging"  the  AETs. 


Troubleshooting  Track  Vehicle  Track/Suspensio  i  Systems  at  the 
Organizational  Level 


This  area  of  troubleshooting  by  an  organizational  mechanic  involves 
identifying  the  causes  of  malfunctions  and  determining  the  serviceability  of 
the  various  torsion  bars,  lockout  cylinders,  support  rollers,  road  wheels 
and  arms,  and  other  components  of  tracked  vehicle  track/suspension  systems. 


Troubleshooting  of  track/suspension  systems  involves  direct  observation 
of  trouble  symptoms  such  as  leaking  seals  and  cylinders,  worn  bushings,  and 
a  thrown  track.  This  area  of  troubleshooting,  therefore,  is  not  as  complex 
as  troubleshooting  of  engines,  but  is  identified  as  a  high-priority  area  for 
RPF-AET  substitution  primarily  because  of  extremely  high  AET  costs.  A 
second  selection  factor  is,  due  to  the  cumbersome  nature  of  the  equipment, 
the  inability  of  track/suspension  system  AETs  to  present  a  variety  of 
malfunction  sjrraptoms  without  an  inordinate  amount  of  instructor  set-up  time 
or  requiring  a  large  number  of  AETs,  each  with  a  different  set  of 
malfunctions.  Training  in  this  skill  follows  the  sane  general  rubric 
described  earlier.  If  trouble  exists  in  a  track/suspension  system,  the 
symptoms  are  quite  obvious.  For  example,  misalignment  of  the  road  wheels  or 
center  guides  produces  evidence  of  wear  in  specific  spot:.  Broken  torsion 
bars  or  leaking  shocks  might  cause  the  vehicle  to  sag  tn  one  side.  Since 
the  S3rmptoms  are  more  obvious  and  the  related  causes  fewei  in  number  than  in 
many  other  areas  of  troubleshooting,  the  instruction  for  this  skill  consists 
mainly  of  description  and  observation  of  the  various  symptoms  and  their 
causes.  AETs  are  used  to  demonstrate  soma  of  the  malfunctions,  but  the 
capability  of  an  actual  track/suspension  system  to  present  a  variety  of 
symptoms  is  severely  limited.  As  in  the  previous  section,  the  trainee  is 
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not  able  to  observe  a  large  portion  of  possible  malfunctions  until  he  works 
on  the  performance  examination,  where  he  is  exposed  to  operational  vehicles 
which  exhibit  n  variety  of  symptoms. 

3 .  Removal/Replacement  of  Engines  and  Power  Packs  at  the  DS/GS  Level 

This  is  a  motor  task  involving  disassembly  of  linkages  and  lines 
attached  to  a  wheeled  vehicle  or  tank  engine  or  power  pack  (engine  and 
transmission  as  one  unit),  removal  of  the  o’"?, j ne/power  pack,  its  reinstal¬ 
lation,  and  reassembly  of  linkages  and  I'Cies.  It  is  recommended  as  an  area 
for  RPF-AET  substitution  primarily  because  of  AET  cost.  Entire  vehicles  are 
devoted  to  teaching  this  task  where  alternatives  could  be  devised  to  reduce 
costs. 


Training  on  this  task,  following  the  same  general  outHne  described 
earlier,  consists  first  of  instruction  in  disassembling  the  numerous  fuel 
lines,  electrical  leads,  hoses,  and  throttle  linkages  attached  to  the 
engine.  If  the  engine  alone  is  to  be  remove'.’,  the  clutch  must  be 
disconnected.  In  power  pack  (engine  and  transmission)  removal,  the 
transmission  must  be  disconnected  from  the  propeller  shaft  and  the 
transmission  linkages  disassembled.  Mounting  bolts  are  then  removed.  At 
this  point,  a  sling  is  placed  around  the  power  plant,  and  in  overhead  crane 
or  wrecker  is  used  to  lift  it  from  its  compartuent.  During  installation, 
tht  irocedures  are  reversed.  The  trainees  must  work  in  teems  in  performing 
this  task. 

4 .  Troubleshooting  Turret  Electrical  and  Hydrau Lie  Systems  at  Both 
Organizational  and  DS/GS  Levels 

Troubleshooting,  as  described  earlier,  involves  analysis  of  trouble 
symptoms  and  diagnosis  of  malfunctions  that  cause  the  trouble  symptoms  to 
appear.  As  such,  troubleshooting  is  a  difficult  skill  to  master.  In 
troubleshooting  the  electrical  and  hydraulic  systems  in  the  modern  turret, 
however,  the  task  is  i  ade  even  more  difficult  because  of  the  sophistication 
of  the  two  separate  systems  and  complexities  introduced  when  both  systems 
are  required  to  interact  in  performance  of  a  number  of  functions  within 
certain  turrets.  It  must  also  be  noted  that  from  an  instructional  point  of 
view,  because  of  lack  of  space  and  poor  visibility,  the  turret  is  not  a  good 
teaching  device.  For  these  reasons,  and  because  turrets  are  very  high-cost 
items,  this  area  of  maintenance  was  identified  as  having  high  potential  for 
cost-effective  RPF-AET  substitution. 

At  both  maintenance  levels,  troubleshooting  of  turret  electrical  and 
hydraulic  systems  involves  use  of  a  number  of  tesit  devices  such  as  the 
multimeter  and  LVCT  to  trace  malfunctions.  The  trainee  must  also  be  able  to 
trace  circuit  diagrams  to  know  where  to  start  testing.  When  a  reading  has 
been  taken,  that  reading  must  be  interpreted  and  the  malfunctioning 
component  identified.  At  this  point,  the  organisational  mechanic  removes 
the  component  and  replaces  it  with  a  new  or  rebuilt  one.  The  DS/GS 
repairman  is  further  confronted  with  the  task  of  disassembling  the 
component,  repairing/replacing  the  malfunctic'ning  partfs),  and  reassembling 
the  component.  Emphasis  is  placed  here  on  electrical  and  hydraulic  systems 
because  the  major  systems  in  turrets  are  either  all  electrical  (in  the  M551) 
or  combinations  of  electrical  and  hydraulic  components  (in  the  M60  series 
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and  in  the  liew  XM-l).  Skills  in  working  with  electrical  and  hydraulic 
systems,  therefore,  are  necessary  if  the  tnechanic  is  effectively  to  maintain 
the  current  inventory  of  turrets. 

Following  the  same  training  format  as  the  other  areas  of  instruction, 
training  in  this  skill  area  centers  primarily  around  malfunction  detection 
through  observation  of  readings  on  certain  test  equipment.  This  is 

particularly  true  for  the  electrical  system,  and  the  LVCT  and  multimeter  are 

vital  tools  to  the  turret  mechanic.  By  connecting  the  test  instruments  to 
test  points  associated  with  system  components,  certain  values  wl  ich  register 
on  the  test  equipment  indicate  certain  malfunctions.  In  working  with 
hydraulic  systems,  pressure  measurements  are  important,  as  is  visual 
inspection  for  fluid  leaks. 

Upon  identification  of  the  malfunctioning  component,  the  skill  of 

physically  removing  the  component  and  replacing  it  with  a  new  or  rebuilt  one 
is  called  into  play.  Tool  selection  and  use,  therefore,  is  another 

important  skill  that  is  taught. 

As  mentioned  earlier,  the  DS/GS  repairman  is  then  faced  with  the  task 
of  disassembling  the  component,  going  through  further  troubleshooting 

procedures  to  isolate  and  replace  the  defective  part,  reassembling  the 

component,  and  then  testing  it  to  assure  it  is  functional  and  can  be 

returned  to  stock. 

EXISTING  RPF  ALTERNATIVES 

As  the  present  effort  was  originally  conceived.  Task  3  was  to  involve 
an  assessment  of  RPF  devices  presently  in  use  in  maintenance  training.  Such 
devices,  if  they  could  be  shown  to  be  cost-effective  substitutes,  mi’ht  then 
be  considered  by  the  Army  as  alternatives  to  some  of  the  AET  devices 

currently  in  use,  particularly  those  AETs  of  high  cost  or  low  training 
e  f fectiveness . 

In  point  of  fact,  the  only  RPF  devices  found  to  be  in  current  use  — 
that  might  be  consideroa  as  alternatives  to  AETs  were  the  so-called  flat- 
panel  systems  simulators  (FPSS).  These  devices  are  used  primarily  in 
teaching  system  troubleshooting  tasks  in  maintenance  training  and  in 
teaching  procedural  tasks  for  operator  training.  Such  devices  are  little 
used  for  the  teaching  of  the  motor  and  cognitive  tasks  of  concern  in  the 
present  study,  but  are  beginning  to  be  used  for  some  aspects  of  engine 
troubleshooting. 

With  reference  to  available  information  concerning  the  cost 
effectiveness  of  existing  FPSS  devices,  an  examination  of  the  few  studies 
that  have  been  done  reveals  little  or  no  valid  cost-effectiveness  data.  In 
certain  uses  (e.g.,  with  complex  electronics  and  radar  systems)  the  FPSS  has 
been  judged  cost  effective:  but  largely  on  the  basis  that  the  AETs  are  very 
high-cost  items.  No  credible  data  were  found  to  demonstrate  FPSS  cost  and 


— ^This  statement  is  made  in  the  context  of  the  Army  and  non-Army 
training  courses  surveyed  in  Task  1  and  the  75  manufacturers  surveyed  in 
Task  2. 


19 


training  effectiveness  in  the  vehicle/mechanical  maintenanc'  areas  of 
concern  in  the  present  effort.  1.1 

As  a  further  possible  means  of  developing  cost  effectiveness  data  on 
these  FPSS  devices  in  the  wheel  and  track  vehicle  maintenance  areas,  the 
TRADOC  Systems  Analysis  Activity  (TRASANA)  was  contacted  cmcerning  possible 
applications  of  its  Cost  and  Training  Effectiveness  Analysis  (CTEA)^/  model 
to  such  devices.  TRASANA  reports  no  knowledge  of  CTEA  a))plication  to  FPSS 
devices.  In  similar  fashion,  the  Army  Research  Institute  (ARI)  was 
contacted  concerning  applications  of  the  training  device  :o8t  effectiveness 
determination  procedure  (TRAINVICE)^/  they  have  developed.  This  too  failed 
to  produce  data  concerning  FPSS  devices  in  wheel  aid  track  vehicle 
maintenance. 

Thus,  no  direct  data  base  could  be  assembled  to  substantiate  the 
training  effectiveness  of  the  FPSS  devices  in  wheel  end  track  vehicle 
maintenance  tr.aining,  nor  were  there  any  sound,  systematic  cost 
effectiveness  data  relating  to  FPSS  use  in  any  setting.  With  reference  to 
the  four  high-ir terest  training  areas  identified  herein,  however,  one  must 
view  the  FPSS  devices  as  potentially  useful  devices  for  AET  substitution  in 


—  See,  for  example: 

Biersner,  R.  Attitudes  and  Other  Factors  Related  to  Aviation 
Maintenance  Training  Effectiveness.  CNETS  Study  Report  No.  6-/1;,  December 
1975. 


Biersner,  R.  Observations  of  the  Use  and  Evaluation  of  ECII-LP 
Simulators  for  Aviation  Training.  CNETS  Report  No.  2-76,  October  1976. 

Darst,  H.  Evaluation  of  ECU  Simulator.  Research  Memorandum  No.  9-75, 
U.S.  Army  Transportation  School,  Fort  Eustis,  Va.,  18  December,  1974. 

Finch,  C.  and  O'Reilly,  P.  The  Role  of  Dynamic  Simul  ition  in  Teaching 
Complex  Problem- Solving  Skills  in  Vocational  and  Tech nical  Education . 
Educational  Technology  Research  Series  No.  56,  Educational  Technology 
Publications,  Englewood  Cliffs,  N.J.,  1973. 

McGuirk,  F.,  Pieper,  W.  and  Miller,  G.  Operational  Tryout  of  a  General 
Purpose  Simulator.  Technical  Report  75-13,  U.S.  Air  Foro  Human  Resources 
Laboratory,  May  1975. 

Wright,  J.  and  Campbell,  J.  Evaluation  of  the  ICII  Programmable 

Maintenance  Simulator  in  T~2C _ Organizational  Maintenance  Training.  Naval 

Air  Systems  Command  Report  No,  NADC-75083-40,  15  May,  1975. 

— ^U.S.  Army.  Cost  and  Training  Effectiveness  Analysis  Handbook. 
TRADOC  PAM  71-10  (Draft)”,  Headquarters ,  U.S.  Army  Training  and  Doctrine 
Command,  Fort  Monroe,  Va.,  3  January,  1977. 

3/ 

—  Narva,  M.  Project  Summary:  Training  Device  Concept/Prototype 

Validation  System  (TRAINVICS).  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  Arlington,  Va.,  April  1976. 
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areas  1  and  4  (i.e.,  tr.  ableshooting  engines  and  turret  electrical  and 
hydraulic  systems).  Their  utility  in  area  2  (tr^^  bleshooting  of 
track/suspension  systems)  would  be  less  likely,  and  they  would  likely  have 
none  at  all  in  area  )  (removal /replacement  of  engines  and  power  parks). 

Perhaps  the  biggest  area  of  concern  in  the  use  cf  FPSS  devices  to 
replace  AETs  in  any  of  the  four  high-interest  areas  wou  d  be  their  general 
inability  to  present  certain  hands-on  instruction  n  tasks  such  as 
remove/instal i ,  adjust,  and  other  largely  motor  tasks.  These  actions, 
sometimes  required  during  troubleshooting  and  usually  as  a  consequence  of 
diagnosis  (i.e.,  executing  corrective-action  tasks),  are  viewed  as  important 
by  maintenance  personnel  and  represent  an  important  area  of  AET  capability 
and  attractiveness. 

To  summarize,  '-'PSS  devices  do  represent  al  tern.it  ives  to  AETs  in 
selected  aspects  of  Army  vehicle  maintenance  training.  The  existing  data, 
however,  do  not  fully  support  a  current  recommendation  that  they  be  procured 
for  such  substitution,  though  a  recommendation  that  the  Army  conduct  a 
systematic,  empirical  cost  effectiveness  evaluation  of  their  use  in  vehicle 
maintenance  training  would  be  in  order. 

NEW  RPF  CONCEPTS 

In  considering  new  maintenance  training  devices  of  a  conceptual  nature 
that  might  be  developed  on  an  experimental  basis,  recent  advances  in 
training  technology  must  be  kept  in  mind.  FPSS  devices  are  capable  of 
providing  training  in  troubleshooting  that  has  frequently  been  considered 
effective,  but  as  described  above,  studies  concerning  the  effectiveness  of 
these  devices  are  lacking  in  specific  cost-effectiveness  data.  There  is  no 
question,  however,  that  the  capability  of  these  devices  to  provide 
instruction  in  the  decision-making  and  procedural  processes  involved  in 
troubleshooting  is  far  beyond  that  of  AETs.  A  significantly  greater  number 
of  malfunctions  can  be  presented  to  trainees  in  a  fraction  of  the  time  it 
requires  an  instructor  to  "bug"  actual  equipment.  Numerous  research  studies 
have  indicated,  moreover,  that  reduced  fidelity  devices,  even  such  low- 
fidelity  devices  as  cardboard  mock-ups,  are  very  effective  in  teaching 
decision-making  and  procedural  processes,  i./  These  are  the  subordinate 
skills,  mostly  cognitive  and  internal,  that  are  the  dominant  elements  in  the 
skill  of  troubleshooting. 


—  See,  for  example: 

Cox,  J.,  Wood,  R.  Jr.,  Boren,  L.  ,  and  Thorne,  H.  Functicnal  and 

Appearance  Fidelity  of  Training  Devices  for  Fixed-Procedures _ Tasks . 

Technical  Report  65-4,  Human  Resources  Research  Office,  Alexandria,  Va., 
June  1965. 

Grimsley,  D.  Acquisition,  Retention,  and  Retraining:  Group  Studies 
on  Using  Low-Fidelity  Training  Devices.  Technical  Report  69-4,  Human 
Resources  Research  Office,  Alexandria,  Va.,  March  1969. 

Prophet,  W.  and  Boyd,  H.  Device-Task  Fidelity  and  Transfer  of 
Training:  Aircraft  Cockpit  Procedures  Training.  Technical  Report  70-10, 

Human  Resources  Research  Organization,  Alexandria,  Va.,  July  1970. 


areas  I  and  4  (i.e.,  troub leshoot ine  engines  and  turret  electrical  and 
hydraulic  systems).  Their  utility  in  area  2  (troubleshooting  of 
track/suspension  systems)  would  be  less  likely,  and  they  would  likely  have 
none  at  all  in  area  1  (removal/replacemeni.  of  engines  and  power  packs). 

Perhaps  the  bij’gest  area  of  concern  in  the  use  <f  FPSS  devices  to 
replace  AETs  in  any  of  the  four  high-interest  areas  wou  d  be  their  general 
inability  to  present  certain  hands-on  instruction  n  tasks  such  .;s 
remove/install,  adjust,  and  other  largely  motor  tasks.  These  actions, 
sometimes  required  during  troubleshooting  and  usually  as  a  consequence  of 
diagnosis  (i.e.,  executing  corrective-action  tasks),  are  viewed  as  important 
by  maintenance  personnel  and  represent  an  important  area  of  AET  capability 
and  attractiveness. 

To  summarize,  '<'PSS  devices  do  represent  alternatives  to  AETs  in 
selected  aspects  of  Array  vehicle  maintenance  training.  The  existing  data, 
however,  do  not  fully  support  a  current  recommendation  that  they  be  procured 
for  such  substitution,  though  a  recommendation  that  the  Army  conduct  a 
systematic,  empirical  cost  effectiveness  evaluation  of  their  use  in  vehicle 
maintenance  training  would  be  in  order. 

NEW  RPF  CONCEPTS 

In  considering  new  maintenance  training  devices  of  a  conceptual  nature 
that  might  be  developed  on  an  experimental  basis,  recent  advances  in 
training  technology  must  be  kept  in  mind.  FPSS  devices  are  capable  of 
providing  training  in  troubleshooting  that  has  frequently  been  considered 
effective,  but  as  described  above,  studies  concerning  the  effectiveness  of 
these  devices  are  lacking  in  specific  cost-cffoctiveness  data.  There  is  no 
question,  however,  that  the  capability  of  these  devices  to  provide 
instruction  in  the  decision-making  and  procedural  processes  involved  in 
troubleshooting  is  far  beyond  that  of  AETs.  A  significantly  greater  number 
of  malfunctions  can  be  presented  to  trainees  in  a  fraction  of  the  time  it 
requires  an  instructor  to  "bug"  actual  equipment.  Numerous  research  studies 
have  indicated,  moreover,  that  reduced  fidelity  decicts,  even  such  low- 
fiielity  devices  as  cardboard  nock-ups,  are  very  effective  in  teaching 
decision-making  and  procedural  processes.  These  are  the  subordinate 
skills,  mostly  cognitive  and  internal,  that  are  the  dominant  elements  in  the 
skill  of  troubleshooting. 


—  See,  for  example: 

Cox,  J.,  Wood,  R.  Jr.,  Boren,  L.  ,  and  Thorne,  H.  Functional  and 
Appearance  Fidelity  of  Training  Devices  for  Fixed-Procedures  Tasks. 
Technical  Report  65-4,  Human  Resources  Research  Office,  Alexandria,  Va., 
June  1965. 

Crimsley,  D.  Acquisition,  Retention,  and  Retraining:  Grouu  Studies 
on  Using  Low-Fidelity  Training  Devices.  Technical  Report  69-4,  Human 
Resources  Research  Office,  Alexandria,  Va.,  March  1969. 

Prophet,  W.  and  Boyd,  H.  Device-Task  Fidelity  and  Transfer  of 
Training:  Aircraft  Cockpit  Procedures  Training.  Technical  Report  70-10 , 
Human  Resources  Research  Orgaaiz  ttion,  Alexandria,  Va.,  July  1970. 
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Instruction  in  the  internal  aspects  of  diagnosis  and  troubleshooting  is 
important,  but  training  in  the  ha  ids-on  aspects  of  these  maintenance  tasks 
should  also  be  presented.  Most  of  the  new  devices  discussed  in  this  section 
of  the  report  make  an  attempt  to  bridge  the  gap  between  these  two  conceptual 
positions  and  provide  training  in  both  areas. 

A  hybrid  training  device,  i.e.,  a  combined  FPSS  and  three-dimensional 
mockup,  is  one  possible  general  solution  to  the  hands-on/simulation  dilemma. 
Combining  the  capability  of  the  FPSS  device  to  provide  instruction  in 
diagnostic  aspects  of  maintenance  with  the  ability  of  a  three-dimensional 
mock-up  to  provide  a  variety  of  hands-on  experiences,  would  extend  the  range 
of  maintenance  tasks  th  t  could  be  taught  far  beyond  that  present  in  current 
FPSS  devices  or  AETs.  This  would  generate  cost  savings  in  at  least  two 
categories,  the  first  being  decreased  dependence  on  expensive  AETs.  The 
second  and  more  significant  category  of  potential  cost  savings  would  arise 
from  developing  improved  maintenance  skills  within  the  trainees,  thus 
increasing  their  efficiency  and  reducing  equipment  downtime.  The  devices 
suggested  herein  would  allow  repeated  practice  on  a  wider  range  of  tasks 
than  is  possible  in  present  training,  and  would  play  a  significant  role  in 
these  two  areas  of  cost  avoidance. 

The  conceptual  descriptions  of  new  RPF  devices  that  follow  are  grouped 
in  terms  of  the  high-interest  training  areas  previously  discjssed.  Two 
devices  are  presented  for  area  1,  one  device  for  area  2,  one  for  area  3,  and 
four  devices,  or  options,  are  presented  for  area  4. 

Troubleshooting  Engines  and  Related  Systems  at  the  Organizational  Level 

Two  different  types  of  devices  will  be  discussed  in  relation  to 
teaching  this  skill.  Device  1  is  a  microprocessor-controlled  mock-up  engine 
with  an  attached  FPSS,  while  Device  2  is  a  three-dimimsional  mock-up  of  a 
basic  engine  block  to  which  would  be  attached  electronic  modules  that  are 
replicas  of  each  of  the  various  engine  components.  Each  of  these  devices  is 
discussed  in  more  detail  in  the  following  paragraphs. 

Device  1 .  This  device  would  be  composed  of  two  separate  components,  a 
three-dimensional  mock-up  engine  an  an  FPSS.  These  components  would  be 
interconnected  electrically  so  that  ae  engine  would  be  under  control  of  the 
simulator.  Thus,  when  the  trainee  performs  a  task,  the  results  of  that 
task,  indicated  by  various  symptoms  or  readings,  would  be  displayed  on  the 
simulator  panel.  ITie  two  major  components  would  be  located  in  close 
physical  proximity,  perhaps  on  an  L-shaped  table.  Figure  1  illustrates  this 
concept . 

The  engine  portion  of  the  device  would  be  a  three-dimensional 
representation  of  an  engine  with  its  components  such  as  starter,  generator, 
carburetor,  etc.  In  order  to  accommodate  for  the  differences  in  design  and 
operation  of  each  engine  type,  a  different  mock-up  engine  ani  simulator 
would  need  to  be  manufactured  for  gasoline,  diesel,  and  raultifuel  engines. 
A  variety  of  hands-on  tasks  such  as  adjusting  the  fuel  mixture  and  setting 
the  idle  speed  would  be  practiced  on  this  device,  but  the  simulated  removal 
and  replacement  of  major  components  would  be  accomplished  by  pressing  a 
button  on  the  simulator.  This  action  would  indicate  to  the  computer  that 
the  component  had  been  replaced. 
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A  high  degree  of  simulation  of  engine  operation  could  be  attained 
through  making  the  engine  "run"  by  means  of  an  electric  motor.  Many 
functions  could  be  simulated,  including  the  capability  to  make  the  engine 
vibrate  by  means  of  a  weighted  wheel  in  order  to  simulate  missing,  rough 
idle,  etc.  The  computer-controlled  electric  motor  could  also  drive  elements 
which  could  automatically  cause  the  fuel  pump  to  fail,  the  choke  to  stick, 
or  create  other  operational  malfunctions. 

The  engine  configuration  would  include  all  the  external  components  and 
linkages,  including  starter,  generator,  accelerator  linkage,  etc.,  of  an 
actual  c.igine.  This  would  allow  the  trainee  to  hook  up  simulated  test 
equipment,  make  adjustments,  and  complete  most  of  the  tasks  performed  on  a 
real  engine  short  of  the  actual  remove/replace  tasks.  The  internal 
components  of  the  engine  would  be  pictured  through  the  visual  capabilities 
of  the  FPSS. 

The  technology  for  the  FPSS  component  of  this  proposed  device  is  in 
existence  on  a  variety  of  trainers  currently  available  from  training  device 
manufacturers.  This  would  include  the  capability  to  present  error  messages, 
display  pictures  of  engine  components,  display  diagrams  of  engine-related 
fuel  and  electrical  systems,  provide  test  points  and/or  buttons,  and  provide 
remove/replace  buttons.  Simulated  test  equipment  such  as  a  tach/dwell 
meter,  LVCT,  and  multimeter  would  also  be  included,  as  well  as  functional 
dashboard  instruments.  The  entire  device  would  be  controlled  from  an 
instructor's  control  panel  incorporated  at  a  convenient  location  on  the 
FPSS. 


In  providing  instruction  in  ne  troubleshooting,  the  device  would 
need  to  display  a  variety  of  -  action  symptoms.  For  example,  if  a 
carburetor  is  set  too  lean  a  viL._c  on  is  caused  by  fuel  starvation.  This 
vibration  could  be  simulated  by  a  weighted  wheel  driven  by  an  electric 
motor.  A  fuel  mixture  needle  would  be  provided  on  the  mock  carburetor,  and 
the  trainee  would  "adjust"  the  fuel  mixture  as  he  would  on  the  real  engine, 
thus  causing  the  simulated  engine  vibration  to  disappear.  In  the  same 
manner,  fouled  spark  plugs,  clogged  fuel  filters  or  lines,  and  a  number  of 
other  malfunctions  could  be  simulated.  The  appropriate  maintenance  or 
repair  action  would  be  simulated  by  the  trainee  through  pressing  specific 
repair  or  remove/replace  buttons  on  the  FPSS. 

A  second  general  example  of  malfunction  symptoms  would  be  those  which 
are  indicated  by  various  items  of  automotive  test  equipment.  A  specific 
instance  would  be  improper  point  dwell  as  indicated  by  a  simulated  tach/ 
dwell  meter  located  in  the  FPSS  panel.  When  an  improper  dwell  angle  is 
observed,  the  trainee  would  make  the  proper  distributor  adjustment  on  the 
mock-up  engine.  Other  malfunctions  identified  by  test  equipment  readings 
would  include  improper  engine  timing  and  excessive  resistance  in  spark  plug 
wires . 


Finally,  many  of  the  cues  needed  by  a  mechanic  uo  isolate  malfunctions 
are  cues  provided  by  engine  sounds.  These  include  the  hissing  of  a  leaking 
vacuum  hose,  clicking  or  tapping  noises,  and  a  variety  of  other  sounds.  In 
order  for  sound  cues  to  be  effective  stimuli  for  instruction,  however,  they 
must  be  generated  so  the  trainee  can  locate  and  isolate  them.  Appropriate 
sounds  of  engine  trouble  could  be  reproduced  and  broadcast  through  small 
speakers  placed  at  strategic  locations  throughout  the  mock-up  engine. 
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Repair  or  remnve/replace  buttons  on  the  simulator  paml  would  allow  the 
trainee  to  demonstrate  his  knowledge  of  the  appropriate  response  to  a  sound 
cue . 


Use  of  this  device  in  teaching  troubleshooting  of  engine  and  related 
systems  would  alter  somewhat  the  current  role  of  the  instructor,  and  two 
noticable  changes  would  be  evident.  First,  the  large  block  of  time  now 
required  to  insert  malfunctions  into  AETs  would  be  reduced  to  the  few 
seconds  it  would  take  to  set  the  malfunction  selector  on  the  instructor's 
control  panel.  In  addition,  as  soon  as  the  trainee  identifies  a  malfunction 
and  takes  the  proper  action  via  response  button  or  physical  action,  another 
malfunction  can  be  selected.  The  trainee  can  thus  be  exposed  to  a  large 
number  of  malfunctions  and  responses  in  a  short  period  of  time,  a 
flexibility  and  range  of  instruction  which  is  not  possible  with  an  AET. 

The  second  major  change  in  the  instructor's  role  v/ould  come  from  the 
device's  assumption  of  portions  of  the  direct  instruction.  This  would  free 
the  irstructor  from  a  number  of  the  very  time-consuming,  and  sometimes 
difficult,  duties  of  merely  dispensing  knowledge,  and  allow  him  more  time  to 
work  individually  with  trainees  in  resolution  of  their  specific  learning 
problems . 

The  physical  actions  required  of  the  trainee  during  instruction  on  this 
device  can  readily  be  inferred  from  the  above  discussion.  He  would  take  a 
variety  of  actual  test  readings,  would  feel  and  hear  many  of  the  same  cues 
provided  by  a  real  engine,  and  make  a  number  of  actual  calibrations  and 
adjustments.  He  would  also  be  able  to  complete  a  number  of  troubleshooting 
procedural  sequences  beginning  with  initial  analysis  of  a  trouble  symptom, 
identification  of  the  malfunction,  prescription  of  the  proper  corrective 
procedure,  and  execution  of  the  corrective  action.  This  last  step,  in  some 
cases,  would  involve  physical  manipulation  of  parts  of  the  engine  mock-up, 
while  in  other  cases  the  action  would  be  simulated  through  pressing 
appropriate  repair  or  remove/replace  buttons  on  the  FP'^S. 

Capabilities  for  measurement  of  trainee  performance  and  provision  of 
feedback  to  trainees,  based  on  current  technology,  could  be  built  into  this 
device  and  provide  many  advantages  over  the  AET,  AETs  have  no  performance 
measurement  features  such  as  visual  or  printed  indications  of  correct 
responses  or  errors,  provision  of  error  messages  or  confirmation  of  correct 
action  decisions,  or  measurement  of  the  time  a  trainee  consumes  in 
completing  those  actions  or  decision  processes.  Immediate  knowledge  of 
results  and  printed  records  of  trainee  achievement  would  be  features  of  this 
device  which  would  enhance  both  the  trainees'  movement  through  the  course 
and  the  measurement,  grading,  and  record-keeping  activities  of  the 
instructors . 

When  compared  to  an  AET,  the  support  requirements  for  this  device  would 
be  minimal  and  would  offer  great  savings  in  space  and  other  resource 
requirements.  The  device  would  require  a  small  amount  of  floor  space  and 
only  a  110-volt  AC  power  supply.  There  would  be  none  of  the  fuel  supply  or 
exhaust  emission  requirements  associated  with  AETs,  thus  reducing  both 
energy  usage  and  air  pollution  to  a  fraction  of  current  levels.  In 
addition,  high-noise  areas  created  by  the  location  of  large  nutrbers  of 
actual  engines  in  an  enclosed  shop  would  not  exist,  thus  eliminating  another 
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potential  health  hazard.  A  clean,  air-conditioned  environment  would  enhance 
the  operation  of  the  device,  and  this  would  perhaps  be  the  most  stringent 
support  requirement. 

This  device  would  be  best  suited  for  use  in  institutional  training 
where  the  inexperienced  trainee  might  need  the  visual  system  configuration 
and  logic  displayed  by  the  FPSS.  The  device  would  assist  in  teaching  the 
trainee  how  the  system  functions,  what  it  does,  and  what  actions  he  is  to 
take.  With  that  knowledge  basis,  he  can  be  more  easily  taught  how  to  take 
the  appropriate  maintenance  actions  in  later  segments  of  AIT  or  in  OJT. 

Device  2.  The  basic  configuration  of  this  proposed  device  (see  Figure 
2.)  would  be  a  generic  engine  block.  This  block  would  be  under 
microprocessor  control  and  would  incorporate  an  electric  motor  to  simulate 
various  normal  or  abnormal  running  conditions  as  described  for  Device  1. 

At  appropriate  locations  throughout  the  engine  block,  electrical  plug¬ 
in  points  would  be  provided.  At  these  points,  replicas  of  the  various  fuel 
and  electrical  system  components,  containing  internal  electronic  circuitry, 
would  be  plugged  into  the  engine  bloi  and  .,ome  under  control  of  the 
microprocessor.  For  a  gasoline  engine,  for  -xample,  these  replicas,  or 
electronic  modules,  would  include  such  componeuuS  as  the  starter,  generator, 
fuel  pump,  water  pump,  carburetor,  distributor,  coil,  and  spark  plugs  end 
wires. 

The  generic  engine  block,  therefore,  would  form  the  foundation  for  a 
family  of  engine  troubleshooting  devices.  With  the  block  as  a  core, 
replicas  of  components  of  various  engine  types  (gasoline,  diesel  or 
multifuel)  and  applications  (automobile,  trucks,  heavy  equipment)  w  .d  be 
manufactured.  With  these  modules  attached  to  the  core  block,  the  generic 
device  would  become  a  realistic  representation  of  a  specific  engine.  By 
interchanging  modules  for  different  engine  types,  one  block  could  provide 
instruction  on  a  variety  of  engines  and  eliminate  the  requirement  for  a 
specific  AET  for  each  type  of  engine  studied. 

As  in  the  Device  1  concept,  instruction  provided  by  this  device  would 
involve  both  diagnostic  and  hands-on  tasks.  The  hands-on  tasks  that  could 
be  taught  would  include,  for  example,  the  capability  to  remove  spark  plugs 
and  perform  a  compression  test  on  the  cylinders.  Tne  opening  from  which  the 
spark  plug  was  removed  would  contain  an  electrical  contact  and  the  reading 
on  the  compression  gauge  would  be  triggered  electrically,  but  the  procedure 
would  be  the  same  as  performed  on  an  actual  engine.  Other  examples  would  be 
the  approximate  setting  of  the  engine  timing  by  positioning  the  distributor, 
and  the  precision  timing  of  the  engine  using  a  timing  light.  In  addition, 
actual  contact  points  could  be  provided  in  the  replica  of  the  distributor  to 
allow  practice  of  the  procedure  for  setting  the  point  gap. 

Use  of  this  device  would  also  modify  the  instructor's  role  in  the  same 
manner  as  described  in  the  Device  1  concept  discussion.  To  avoid  repetition 
of  detail;  the  reader  is  referred  to  that  discussion  for  this  information  as 
well  as  that  referred  to  periodically  in  following  paragraphs. 
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The  trainee  interaction  with  this  device  again  can  be  inferred  from  the 
preceding  discussion.  Since  this  device  will  not  incorporate  an  FPSS  as  did 
Device  1,  engine  system  diagrams  and  visual  displays  would  not  be  present. 
The  trainee  would,  therefore,  interact  more  directly  with  the  engine  mock-up 
in  performing  diagnostic  procedures  and  completing  motor  tasks  such  as 
adjusting  the  fuel  mixture  and  setting  the  point  gap.  The  same  performance 
measurement  and  feedback  capabilities  as  described  in  Device  1  could  be 
incorporated  into  this  device. 

The  device's  support  requirements  in  terms  of  floor  space  would  be 
approximately  the  same  as  an  AET,  but  requirements  for  physical  resources, 
especially  energy,  would  be  minimal  as  compared  to  AETs.  One  significant 
support  requirement,  however,  would  be  the  development  of  replicas  of  test 
equipment  that  would  provide  such  readings  as,  for  example,  cylinder 
compression  (in  pounds  per  square  inch),  but  would  utilize  an  electrical 
source. 

As  stated  earlier,  the  organizational  mechanic  can  face  a  bewildering 
arra"  of  vehicle  types  upon  reporting  to  his  unit  assignment.  Moreover,  his 
institutional  training  more  than  likely  was  focused  on  a  relatively  small 
number  of  vehicles;  and  the  transfer  of  that  knowledge  to  other  systems  is 
expected  to  occur  through  OJT.  Because  of  the  flexibility  offered  by  the 
core  engine  block  and  interchangable  module  concept  described  here,  this 
device  is  recommended  for  use  at  the  unit  level  so  that  training  of  new 
maintenance  personnel,  or  retraining  of  experienced  personnel,  can  be 
offered  on  a  device  configured  like  the  equipment  used  in  that  unit.  This 
could  significantly  reduce  the  amount  of  time  the  unit's  crucial  equipment 
is  diverted  from  operational  missions  for  use  in  OJT. 

Troubleshooting  Track  Vehicle  Track/Suspension  Systems  at  the 
Organizational  Level 


In  the  training  of  track  vehicle  and  automotive  mechanics  observed  in 
this  study,  this  skill  is  taught  using  an  entire  tracked  vehicle  as  the 
training  device.  It  is  difficult  to  "bug"  equipment  as  cumbersome  as  the 
track/suepension  system  on  one  of  these  vehicles,  so  much  of  the  instruction 
on  this  task  can  be  offered  only  when  an  operational  vehicle  is  available 
that  exhibits  a  set  of  malfunctions  that  can  be  diagnosed  and  corrected. 
Another  option  present  in  training  is  to  use  an  AET  that  exhibits  a  specific 
set  of  problems  which  are  merely  diagnosed  by  the  trainee  and  not  corrected. 
The  next  group  of  trainees  is  required  to  diagnose  the  same  set  of  problems, 
resulting  in  a  situation  where  trainees  sometimes  know  a  priori  which  set  of 
responses  will  result  in  a  "go"  on  that  segment  of  instruction. 
Furthermore,  the  number  of  malfunctions  that  can  be  introduced  into  training 
is  severely  limited. 

A  device  concept  proposed  for  teaching  this  skill,  therefore,  is  a 
microprocessor-controlled  mock-up  of  the  track/suspension  system  of  a 
tracked  vehicle  including  the  suspension  arms,  road  wheels,  and  tracks. 
This  device,  illustrated  in  Figure  3,  would  be  mounted  on  a  platform. 
Powered  by  an  electric  motor,  rollers  or  continuous  belts  would  drive  the 
tracks  to  simulate  both  horizontal  and  vertical  motion  of  the  suspension 
system  as  if  the  vehicle  were  moving  over  terrain. 
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Track/ Suspension  System  Troubleshooting  Device 


The  instructional  capabllltits  of  this  device  would  allow  the  trainee 
to  observe,  diagnose,  and  correct  a  number  of  malfunctions  such  as  rough 
riding,  pulling  to  the  left  or  right,  sagging  to  one  side,  or  alignment 
problems.  Hie  trainee  would  correct  these  types  of  symptoms  by  pressing  a 
repair  or  remove/ replace  button  located  on  th(^  component  or  on  a  control 
panel.  Simulation  of  such  malfunctions  as  leaks  in  cylinders  or  Idler  wheel 
hubs  would  be  accomplished  by  the  computer  activating  a  "leak"  light  at  the 
appropriate  point  on  the  mechanism.  The  trainee  would  then  show  his  repair 
of  the  leak  by  pressing  a  button  indicating  the  correct  action. 

In  presenting  other  types  of  problems  such  as  a  track  that  has  slipped 
off  a  road  wheel,  the  trainee  would  use  replicas  of  actual  tools,  such  as  a 
pry  bar,  in  correcting  the  problem.  The  track  would  be  made  of  a  material 
that  would  be  lighter  than  an  actual  track,  so  v</ooden  or  ruggedized  plastic 
replicas  of  actual  tools  could  be  used.  Hie  trainee,  however,  would  be 
exposed  to  all  aspects  of  the  repair  procedure  except  the  weight  of  the 
objects  Involved.  Thus,  he  would  know  precisely  what  to  do  when  he  is  first 
confronted  with  that  particular  problem  on  an  item  of  actual  equipment. 

As  ill  previously  described  troubleshooting  device  concepts,  use  of  this 
device  would  cause  the  instructor  to  become  more  of  a  guide  of  learning  than 
a  dispenser  of  knowledge.  In  addition,  the  performance  measurement  and 
feedback  capabilities  of  this  device  would  be  similar  to  those  already 
described. 

Since  this  would  be  a  full-size  mock-up  of  a  track/suspension  system, 
the  floor  space  requirements  for  this  dt;vice  would  be  the  same  as  for  an 
AET.  Other  requirements  would  include  appropriate  tools  and  access  to  an 
electrical  power  supply. 

Be  ;ause  the  track/suspension  systems  of  moot  tracked  vehicles  contain 
many  sinllar  components,  it  is  recommend  ad  that  a  generic  device  be  built 
represe  iting  a  class  of  vehicles  (i.e.,  tanks  or  the  new  infantry  fighting 
vehicles).  Thus,  this  device  would  be  best  suited  for  the  introductory 
training  presented  in  AIT. 

Removal/Replacement  of  Engines  and  Power  ?acks  at  the  DS/GS  Level 

In  the  training  observed  during  this  study,  the  removal  and  replacement 
of  wheeled  vehicle  and  tank  engines  anc  power  packs  was  a  skill  taught 
without  exception  using  an  entire  vehicle  as  the  training  device.  This 
skill  could  be  taught  more  cost  effectively  using  a  mock-up  of  the 
engine/ power  pack  (power  plant)  portion  of  the  vehicle  rather  than 
dedicating  an  entire  vehicle  to  the  task. 

The  device  concept  suggested  for  teaching  this  skill,  therefore,  is  a 
mock-up  of  the  power  plant  compartment  of  a  tank  or  a  truck,  incorporating  a 
replica  of  the  power  plant.  Figures  4  ard  5  illustrate  this  concept.  The 
power  plant  replica  would  be  a  hollow  metal  shell,  shaped  and  weighted  like 
an  actual  power  plant.  It  would  provide  the  actual  hands-on  experiences  and 
would  incorporate  all  the  attachments  and  accessories  of  the  actual  power 
plant,  allowing  for  use  of  real  tools  and  hoist  equipment  in  the  instruction 
of  this  task. 
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This  relatively  straightforward  motor  task  is  outlired  in  detail  in  the 
TM  for  the  particular  vehicle,  and  trainees  are  required  to  complete  the 
sequential  procedures  as  described.  The  sequence  in  which  the  accessories 
are  unhooked  or  removed,  the  manner  in  which  the  sling  is  placed  around  the 
engine,  and  the  precautions  taken  in  lifting  ^rhe  power  plant  from  the 
compartment  are  described  fully  in  the  TM.  Since  the  trainees  perform  this 
t isk  "by  the  numbers"  from  the  TM,  the  instructor's  role  in  teacl  ing  this 
t isk  is  minimal  and  would  not  change  if  this  device  were  used.  The  trainee, 
also,  would  interact  with  the  device  in  exactly  the  same  manner  as  with  the 
AIT. 


Because  this  task  comprises  a  series  of  straightforward  motor  tasks,  no 
pjrticular  lerformance  measurement  or  feedback  capabilities  are  required  of 
tie  proposed  device.  Since  the  device  is  a  mock-up  of  only  a  portion  of  the 
A^T,  it  would  require  a  smaller  working  space,  and,  with  the  exception  of 
tie  hoist  apparatus  needed,  would  consume  no  energy  resoirces. 

A  generic  device  representing  tank  power  pack  compa'tments  and  another 
representing  truck  engine  compartments  could  provic'e  an  appropriate 
introductory  level  of  training  for  this  basic  maintenance  task.  The 
proposed  device,  therefore,  would  be  best  suited  for  use  in  institutional 
training. 

Troubleshooting  Tun et  Electrical  and  Hydraulic  Systems  at  both 
Organizational  and  I S/GS  Levels 

As  described  ii  an  earlier  section  of  this  repor  ,  the  difficulties 
encountered  by  a  trainee  in  performing  troubleshooting  t isks  are  compounded 
by  the  complex  nature  of  the  many  turret  systems  whi  h  incorporate  both 
electrical  and  hydraulic  components.  In  addition,  the  high  cost  of  turrets 
and  their  general  ineffectiveness  as  training  devices,  due  primarily  to  lack 
of  working  space  and  visibility  problems,  have  also  been  cited. 

Four  turret  training  devices,  all  designed  arou'  d  a  single  device 
concept  but  possessing  different  physical  characteristics,  are  described  in 
the  following  paragraphs.  In  this  discussion,  the  gen  ric  device  concept 
(pictured  in  Figure  6)  is  presented,  followed  by  physi  al  descriptions  of 
the  four  alternative  devices.  The  instructional  capab  lities  and  related 
functions  of  the  devices  are  then  discussed. 

The  basic  device  concept  around  which  each  of  the  f  )ur  alternatives  is 
developed  is  (hat  of  a  mock-up  turret  incorporating  a  flat-panel  system 
simulator  within  the  structure.  The  electrical  and  hydra  ilic  components  and 
and  circuitry  of  the  various  turret  systems,  color -coded  for  easier 
differentiation  by  the  trainee,  would  be  represented  in  their  appropriate 
locations.  Each  device  would  house  functional  representations  of  such 
componenis  as  the  commander's  panel  and  gunner's  control  switches. 

To  maximize  each  device's  instructional  capabili' ies ,  the  physical 
structurj  must  be  designed  to  overcome  the  space  and  visibility  weaknesses 
of  the  turret  AET.  The  basic  difference  among  the  four  devices  described 
below  is  the  manner  in  which  the  visibility  problem  is  o'-ercome.  Tlie  first 
alternative  utilizes  an  open  skeleton  framework,  whilt  the  latter  three 
devices  incorporate  transparent  side  panels  which  W(  uld  fit  into  the 
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skeleton  framework.  In  this  manner,  the  internal  systems  and  components  of 
the  turret  would  be  visible  from  both  inside  and  outside  the  structure, 
allowing  for  maximum  flexibility  in  individual,  small-grcup,  or  large-group 
instruction. 

To  alleviate  the  problem  of  lack  l  working  space,  it  is  recommended 
that  all  four  mock-ups  be  built  larger  than  actual  si  e.  In  self-paced 
instruction,  space  must  be  provided  for  the  instructor  and  trainee(s)  to 
work  together  and  have  access  to  tools,  test  equipment,  and  TMs .  Space 
within  a  turret  AET  for  that  type  of  interaction  is  severely  limited,  and 
larger-than-scale  construction  of  the  training  devices  would  greatly  enhance 
the  instructional  process. 

The  FPSS  incorporated  within  each  device  is  an  ettremely  important 
feature.  Due  to  the  complexities  of  turret  systems,  particularly  the 
electrical  systems,  clearly  presented  functional  diagrams  of  the  systems  and 
their  flow  and  logic  are  necessities  in  turret  troubleshooting  training.  In 
fact,  this  was  the  only  area  of  maintenance  training  observed  in  this  tudy 
where  the  instructors  were  eager  to  discuss  RPF  alternatives  to  AETs. 
Recognizing  the  interaction  between  the  intellectual  ability  levels  among 
^^•ainees  and  the  sophistication  of  modern  turret  systems,  those  instructors 
cited  their  need  for  device  alternative  to  AETs  that  would  facilitate 
instruction  of  turret  troubleshooting. 

The  instructional  features  of  the  ’^PSS  for  turret  tre ublishooting  would 
be  similar  to  those  described  in  earlier  troubleshooting  discussions.  Ttiis 
would  include  the  display  of  system  diagrams,  provisions  of  buttons  to 
indicate  the  test,  repair,  or  removal/replacement  of  a  component,  and 
provision  of  error  messages  or  confirmation  of  coiiect  retponses.  Dials  and 
gauges  would  also  be  displayed  as  needed.  The  FPSS  would  also  contain  an 
instructor's  control  panel  for  selection  of  malfunctions  and  for  making  the 
various  trainee  performance  measurements  cited  earlier. 

Based  on  this  general  description,  additional  deta  1  on  each  of  the 
tour  alternative  devices  is  presented  below. 

Alternative  1.  This  device  would  consist  of  a  skelt ton  rame  forming 
the  shape  of  a  turret.  Attached  to  the  open  framework  in  their  actual 
locations,  would  be  mock  representations  of  the  circuitry  and  components  of 
the  various  systems.  The  trainee  would  do  work  in  the  turret  while  being 
observed  and  assisted  by  the  instructor  from  either  inside  or  outside  the 
open  framework. 

The  systi’m  components  in  this  version  of  the  device  w  >uld  be  functional 
in  that  the  t  ainee  would  use  his  LVCT  and  multimeter  in  erforming  various 
tests.  P.eraov. ll/replacement  of  a  component  could  be  inaic.  ted  by  pressing  a 
button  on  the  FPSS  panel  or  by  actual  manipulation  of  toe  components,  if 
they  could  be  made  of  ruggedized,  long-lasting  material. 

Alternative  2.  This  version  of  the  turrei  would  contain 
interchangeable  transparent  side  panels  which  would  fit  Into  the  skeleton 
framework.  Laminated  within  the  panels  would  be  v  sual  color-coded 
representations  of  the  components  and  circuitry  of  the  systems,  and  the 
device  would  include  a  different  set  of  panels  for  i  ach  system  being 
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studied.  To  provide  instruction  regarding  a  particular  system,  the  instruc¬ 
tor  would  place  the  transparent  side  panels  in  the  turret  skeleton  in  ini¬ 
tial  phases  of  training,  but  could  put  multiple  sets  of  panels  in  place  in 
later  phases  of  training  so  the  trainee  could  do  work  associated  with 
several  systems  at  the  same  time.  The  side  panels  in  this  option  would  be 
inert,  and  the  trainee  would  do  all  the  troubleshooting  work  on  the  FPSS. 
This  would  require  a  simulated  LVCT  and  multimeter  be  built  into  the  FPSS. 

Alternative _ This  would  be  essentially  the  same  as  Alternative  2 

except  the  interchangeable  transparent  side  panels  would  also  be 
interactive.  Wiring  would  be  incorporated  within  the  panels  so  the 
components  could  be  controlled  by  the  computer  and  the  trainee  could  make 
tests  with  the  meters  at  the  actual  locations.  Since  the  panels  would 
contain  live  test  points,  the  actual  LVCT  and  multimeter  could  be  used,  thus 
allowing  for  their  removal  from  the  FPSS  in  this  version  of  the  device. 

The  engineering  required  to  provide  interactive  panels  that  could  be 
removed  and  replaced,  however,  would  be  extensive.  Contact  points  and 
wiring  would  need  to  be  incorporated  within  both  the  panels  and  the  main 
structure,  increasing  their  physical  complexity. 

Alternative  4.  In  this  version  the  transparent  side  panels  would  not 
be  interchangeable,  but  would  be  interactive.  Rather  than  a  set  of  panels 
for  each  system,  in  this  version  all  the  systems  would  be  shown  in  coloi 
coded  representations  within  fixed  transparent  side  panels.  The  components 
of  the  systems  represented  within  the  panels  would  be  wired  so  they  would  be 
under  control  of  the  computer  and  live  test  points  could  be  located  in  the 
same  positions  as  in  the  actual  turret.  Since  the  panels  in  this  version 
would  be  fixed  in  place  within  the  skeleton  framework,  their  engineering 
would  be  less  complex  than  in  Alternative  3. 

Th'  inplructional  capabilities  of  all  four  turrets  would  be  essentially 
the  same,  and  the  four  are  designed  to  accomplish  many  common 
troubleshooting  tasks.  Through  the  FPSS  control  panel,  the  instructor  would 
select  a  malfunction  in,  for  example,  the  electrical  system.  The  trainee 
would  attach  either  his  LVCT  or  multimeter  at  appropriate  points  in  the 
circuits,  determine  the  reading  and  interpret  its  significance.  (An 
exception  to  this  would  be  present  in  Alternative  2,  whose  side  panels  would 
be  inert,  and  the  LVCT  and  multimeter  readings  would  appear  on  the  FPSS 
panel).  If  removal  and  replacement  of  a  component  is  indicated,  the  trainee 
would  press  the  appropriate  remove/replace  button  on  the  FPSS,  which  would 
either  confirm  his  correct  decision  or  present  an  error  message.  If  the 
decision/action  was  correct,  the  instructor  would  select  another 
malfunction.  If  the  trainee  was  in  error,  he  would  attempt  to  rectify  it. 
Throughout  this  procedure,  the  FPSS  would  be  recording  the  triiim  o’s  actions 
and  measuring  the  time  required  in  their  performance. 

Again,  as  described  in  earlier  troubleshooting  sections,  the 
instructor's  traditional  role  would  be  modified  if  this  device  were  used  in 
troubleshooting  training.  The  trainee's  interaction  with  this  device  would 
be  much  the  same  as  cited  in  other  troubleshooting  discussions,  except  that 
he  would  be  surrounded  by  a  three-dimensional  representation  of  the  actual 
turret. 
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Since  these  devices  would  be  larger  than  an  actual  turret,  some 
additional  floor  space  would  be  required.  An  electrical  power  source  would 
also  be  a  necessity. 

These  devices  could  be  used  in  both  institutional  and  unit  training. 
They  would  provide  excellent  introductory  training  in  AIT,  and  could  be 
quite  useful  in  skill  improvement  training  at  the  unit  I’vel. 

Regardless  of  which  configuration  of  this  device  .lould  be  used,  the 
ability  of  the  instructor  to  work  with  the  trainee  fron  inside  or  outside 
the  turret,  the  turret  being  larger  than  actual  size  for  easier  access,  and 
the  addition  of  a  systems  simulator  would  enable  the  device  to  provide 
training  in  a  wider  range  of  malfunctions  and  problem  s; tuations  in  a  more 
cost-  and  training-effective  manner  than  existing  tu *ret  AETt .  These 
features  could  have  a  positive  impact  on  the  training  of  urret  mechanics  in 
both  institutional  and  unit  settings. 

EVALUATION  OF  TRAINING  EFFECTIVENESS 

The  purpose  of  Task  5  was  to  develop  a  methodology  for  evaluating  the 
relative  training  effectiveness  of  the  various  RPF  devices  identified  in 
Tasks  3  and  4.  The  methodology  is  to  provide  for  a  comparison  of  the 
training  effectiveness  of  each  device  with  a  corresponding  AET,  and  with  an 
alternative  device  when  available.  When  implemented,  the  methodology  will 
produce  data  which  can  be  used  not  only  to  assess  the  effectiveness  of  each 
device  for  achieving  training  objectives,  but  more  importantly,  their 
effectiveness  in  preparing  trainees  to  perform  on  the  job.  In  addition  to 
serving  these  purposes,  the  data  will  provide  needed  input  for  analyses  of 
training  cost  effectiveness. 

The  present  section  presents  an  overview  of  the  evaluation  methodology 
that  was  developed  during  Task  5.  It  provides  (1)  an  instructional  context 
for  the  evaluations;  (2)  a  discussion  of  the  pertinent  facets  of  the  context 
as  they  apply  to  the  present  development  of  an  evaluaticn  methodology;  and 
(3)  a  description  of  the  technical  capabilities  that  must  be  represented 
among  personnel  conducting  the  evaluations. 

Appendix  D  presents  the  evaluation  methodology  in  s  ibstantial  detail. 
That  Appendix  is  intended  for  use  as  a  guide  in  the  event  lal  implementation 
of  the  evaluations.  As  a  guide,  it  develops  a  framework  cor  conducting  the 
evaluations  that,  with  suitable  adaptations,  can  be  applied  to  each  of  the 
devices  as  it  is  used  in  each  of  the  various  target  training  programs. 

Instructional  Context  for  the  Evaluation 


The  evaluation  of  a  training  device  can  be  viewed  wit  lin  the  context  of 
training  course  development.  Various  branches  of  the  mil  tary  have  jointly 
developed  a  set  of  procedures,  commonly  referred  to  as  ISD  which  includes 


—  Interservice  Procedures  for  Instructional  Systems  Development. 
TRADOC  Pamphlet  350-30,  Headquarters;  U.S.  Army  Training  and  Doctrine 
Command,  Ft.  Monroe,  Va.,  1  August,  1975. 
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seven  major  steps  to  be  followed  in  conceiving,  developing,  implementing, 
and  evaluating  a  training  course.  These  steps  will  not  be  detailed  here. 
Instead,  eight  related  or  derivative  procedural  steps  of  particular  concern 
in  training  course  evaluations  will  be  identified.  Five  of  these  steps  will 
be  discussed  at  some  length  in  the  section  that  follows,  and  then  treated  in 
considerable  detail  in  Appendix  D. 

The  eight  procedural  steps  for  evaluation  are  as  follows: 

1.  Statement  of  the  evaluation  problem.  The  evaluation  problem  as  it 
applies  to  the  particular  case  at  hand  must  state  clearly  the  issues  to  be 
resolved,  and  in  such  a  manner  that  requirements  tor  the  remaining 
procedural  considerations  are  clearly  determined. 

2 .  Analysis  of  cognitive  and  manipulative  skills  to  be  taught. 
Specific  training  objectives  for  the  course  of  concern  inust  be  stated  in 
terms  of  those  knowledges  and  skills  required  for  job  performance,  and  for 
which  trainees  are  to  be  prepared  in  the  course. 

3.  Development  of  the  training  course.  This  step  encompasses  the 
design  of  the  instructional  process  whereby  course  objectives  are  to  be 
achieved. 


4.  Selection  of  criterion  measures.  Psychometrically  sound  measuring 
instrumencs  and  procedures  must  be  available  for  determining  as  objectively 
as  possible  the  extent  to  which  course  objectives  are  achieved  by  trainees. 

5.  Design  of  the  evaluation.  All  independent  variables  of  concern  in 
the  evaluation  must  be  identified  and  their  roles  i  ist  be  implemented  in 
such  a  way  that  the  data  to  be  collected  reflect  clearly  the  effects  of  each 
such  variable.  In  addition,  provisions  must  be  made  for  controlling  the 
effects  of  variables  that  are  not  in  themselves  of  concern,  but  which,  if 
not  controlled,  can  prevent  a  clear  identification  of  effects  of  the 
independent  variables, 

6.  Collection  of  data.  This  step  must  provide  for  the  administration 
of  all  measuring  instruments.  The  sources  of  data  must  be  specified  and  a 
schedule  for  the  collection  established. 

7.  Analysis  of  data.  The  conceptual  structure  for  examining  data 
should  be  clear  at  the  outset,  and  a  basic  analytic  schema  established. 

8 .  Integration  of  findings  vis-a-vis  the  evaluation  problem.  The 
procedures  involved  in  this  step  are  to  be  directed  toward  resolving  issues 
which  gave  rise  to  the  evaluation  effort,  using  the  information  obtained 
from  that  effort. 

Procedural  Considerations  Applicable  to  the  Present  Effort 

Three  of  the  procedural  steps  above  go  beyend  the  scope  of  the  present 
effort.  Specif  ically ,  Steps  2,  3,  and  8  will  be  part  of  the  final 
evaluation  effort,  and  the  previously  referenced  ISD  procedures  provide 
adequate  guides  for  their  completion.  Hence,  they  will  not  be  discussed 
further  in  this  report  except  for  general  considerations  concerning  them  as 
they  apply  to  the  remaining  steps. 
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Steps  1  and  4-7,  or  certain  aspects  of  them,  are  of  particular  concern 
in  this  project.  They,  too,  are  included  in  ISD  procedures,  but  Task  5  of 
this  project  calls  for  adapting  these  steps  to  the  unique  problems  of 
evaluating  the  maintenance  training  devices  identified  in  this  report.  The 
remaining  part  of  this  subsection  will  discuss  these  five  steps  in  order, 
focusing  upon  their  adaptations  to  the  evaluation  of  maintenance  training 
devices . 

Statement  of  the  problem.  This  step  has  two  facets.  The  first  is  to 
provide  a  guide  for  deriving  the  remaining  procedural  steps;  and,  second,  by 
its  derivation,  it  should  provide  a  framework  for  interpreting  evaluative 
data  and  for  determining  adaptive  actions  based  on  the  data.  The  second 
facet  is  covered  in  Section  I  in  discussions  of  the  need  for  better,  cost- 
effective  maintenance  training,  the  military  problem  of  providing  such 
training,  and  the  research  problem  prompted  by  the  need  and  the  military's 
reaction  to  it. 

The  first  facet  of  the  problem  statement  is  a  clear  definition  of  the 
issues  to  be  resol /ed,  stated  so  as  to  establish  recuirements  for  the 
remaining  procedural  steps.  For  present  purposes,  the  problem  is  to 
determine  the  trailing  effectiveness  of  certain  alternative  maintenance 
training  devices  relative  to  corresponding  AETs,  and,  in  some  cases,  to 
alternative  devices.  The  problem  requires  that  research  procedures  be 
est.tblished  for  answering,  or  otherwise  resolving,  seven  questions  regarding 
training.  These  questions  are: 

1.  What  kinds  of  measures  can  be  used  that  are  indicative  of  training 
ef fectivene  58? 

2.  From  what  50urces,  under  what  conditions,  and  when  should  these 
measures  be  obtained? 

3.  How  are  the  measures  to  be  analyzed  so  as  to  provide  coLiparative 
indices  of  RPF  and  AET  training  effectiveness? 

4.  Is  the  ef f‘-ctiveness  of  a  device,  including  AET,  dependent  upon 
certain  instructional  modes? 

5.  Is  the  effectivness  of  a  device  per  se,  or  its  use  in  alternative 
instruction.il  modes,  dependent  upon  certain  student  characteristics 
such  as  aptitude  level  or  amount  of  relevant  experience? 

6.  Because  instructors  may  vary  in  the  effectiveness  with  which  they 
use  various  devices  in  training,  how  are  instructor  effects  to  be 
provided  for  in  the  analysfi^? 

7.  Because  training  sites  may  differ  in  ways  affecting  device  use  and 
training  value,  how  are  the  effects  of  these  differences  to  be 
determined  or  otherwise  provided  for  in  the  analyses? 

The  subsections  which  follow  address  these  questions  generally  through 
discussions  of  the  remaining  procedural  steps  identified  earlier.  For 
detailed  discussions  of  the  issues  involved,  see  Appendix  D. 
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Selection  of  criterion  measures.  The  primary  concern  In  evaluating 
training  devices  Is  the  adequacy  of  on-the-job  performance  of  graduates  of 
training  programs  using  the  devices.  It  Is  necessary  therefore  that  job 
competence  of  graduates  be  assessed,  and  In  a  manner  that  reveals  the 
adequacies,  and  lacks  thereof,  of  devices  used  In  preparing  t'iiem  for  unit 
responsibilities.  Adequacy  of  job  performance  Is  only  partially  determined 
by  the  fulfillment  of  AIT  objectives,  l.e.,  the  achievement  of  cognitive  and 
hands-on  skills  per  se.  Such  skills  are  necessary,  but  they  must  be 
integrated  into  appropriate  judgments  and  actions  as  determined  by  the  cues 
and  demands  of  real-world  situations.  In  addition,  the  graduate  trainee  is 
expected  to  progress  on  the  job  beyond  the  achievement  levels  attained  in 
AIT.  Hence,  indicators  of  job  performance  must  tap  capabilities  and  actions 
that  cannot  be  observed  during  AIT. 

Time  constraints  require  that  such  assessments  be  made  soon  after 
prototype  training  devices  become  available,  for  decisions  regarding  the 
procurement  of  additional  devices  must  be  made  wltnout  undue  delay.  Hence, 
job  performance  measures  that  are  revealing  of  competence,  and  that  can  be 
obtained  relatively  soon  after  training,  must  be  used. 

A  second  set  of  measures,  a  group  administered  during  and  immediately 
upon  completion  of  AIT,  should  Include  knowledge  or  cognitive  skills,  and 
manipulative  or  hands-on  skills.  Such  measures  can  reveal  specific 
strengths  and  weaknesses  of  training  programs  according  to  attainments  of 
particular  training  objectives.  These  indicators  would  not  only  reveal 
trainee  achievement  during  training  as  it  relates  to  device  use,  but  to  the 
extent  that  such  achievement  is  predictive  of  later  job  performance,  they 
provide  data  within  the  time  constraint  that  can  be  used  in  decisions 
regarding  device  procurement. 

To  be  useful  for  evaluating  training  devices,  measures  obtained  on  the 
job,  or  during  or  at  the  end  of  AIT,  should  focus  upon  aspects  of 
achievement  which  are  relevant  to  device  design  and  utilization.  The 
devices  to  be  evaluated  will  be  used  for  only  certain  segments  (e.g., 
gasoline  engine  maintenance)  of  overall  AIT  training.  Furthermore,  among 
the  training  objectives  concerned  with  these  particular  segments,  the 
achievement  of  only  certain  ones  can  reasonably  be  expected  to  be  enhanced 
by  given  devices.  Device-relevant  objectives  should  be  identified,  and 
provisions  should  be  made  for  separating  measures  of  their  attainment  from 
measures  concerned  with  objectives  not  related  to  Che  purposes  implied  by 
device  design  and  use. 

A  different  type  of  measure,  one  of  user  acceptance  of  the  various 
devices,  will  also  be  needed  for  device  evaluation.  The  focus  for  such 
measures  should  be  upon  the  satisfaction  of  trainees  and  Instructors  with 
training  devices  used,  and  upon  job  supervisors'  satlsfacti  )n  with  trainees 
after  training.  Because  the  value  of  a  device  depends  tc  a  great  extent 
upon  how  it  is  used,  acceptance  measures  should  reflect  satisfaction  with 
the  relevant  segments  of  the  training  program  as  well. 

Device  acceptance  by  trainees,  instructors,  and  Vy  implication, 
supervisors,  is  of  concern  for  two  reasons.  First,  it  is  important, 
although  perhaps  not  crucial,  for  morale  that  users  of  given  programs  and 
media  see  them  as  valuable,  and  their  roles  in  them  as  satisfying.  Second, 
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in  the  event  that  a  device  is  cr  could  be  trainin;;  effective,  but  generally 
is  not  seen  as  satisfactory  by  groups  of  users,  establishing  its 
nonacceptability  would  provide  a  signal  for  possible  device  design  changes 
or  for  needed  managerial  actions  directed  toward  educating  the  users 
regarding  the  device's  value. 

Still  another  kind  of  measure  that  can  aid  in  the  evaluation  of  a 
device  concerns  characteristics  and  utilization  of  the  device  itself. 
Information  sought  by  these  measures  includes  amount  and  nature  of  device 
use;  instructors'  and  trainees'  critiques  of  the  device;  the  level,  depth, 
and  comprehensiveness  of  training  attempted  with  the  devices;  adequacy  of 
software;  amount  and  nature  of  instructor  and  trainee  effort  required  to  use 
the  device;  equipment  set-up  time;  etc. 

Design  of  the  evaluation.  The  training  di!vice  is  the  independent 
variable  of  primary  concern.  Hence,  the  focus  foi  the  evaluation  is  on  the 
comparisons  of  the  training  effectiveness  of  various  RPF  devices  with 
corresponding  AETs ,  and  when  they  are  availai>le,  the  comparisons  of 
alternative  RPF  devices.  Important  to  these  comparisons  are  the  effects  of 
using  the  devices  (including  AETs)  in  lock-step  as  opposed  to  self-paced 
programs.  In  addition,  the  effectiveness  of  ea(  h  device  for  trainees  of 
different  aptitude  levels  or  amounts  of  relevant  experience  is  of  concern. 
Hence,  data  should  be  collected  that  permit  meaningful  conparisons  of  device 
effectiveness  separately  by  instructional  mode,  and  by  these  trainee 
characteristics. 

These  comparisons  require  that  separate  training  groups  be  established 
for  each  device-instructional  mode  combination.  Within  each  group,  all 
levels  of  trainee  characteristics  are  to  be  represented.  Furthermore, 
possible  confoundin;?  influences  on  achievemei  t  due  to  differential 
instructor  effectiveness  should  be  avoided  as  described  in  Appendix  D. 

It  is  especially  important  that  training  programs  using  a  giv m  device 
be  standardized,  so  that  achievement  and  other  dat  a  relevant  to  ea  :h  device 
are  comparable.  The  only  variations  from  strict  training  program 
standardization  that  can  be  permitted  are  those  necessary  for  providing 
lock-step  vs.  self-paced  instruction,  and  those  necessary  due  to  differences 
in  device  instructicial  capabilities.  In  either  case,  however,  the  program 
for  a  given  instructional  mode  or  device  should  be  standaid. 

Even  so,  evaluition  data  are  to  be  gathered  from  different  training 
sites,  and  even  though  program  standardization  is  attained,  local  training 
philosophies  and  other  similar  influences  can  reider  data  noncomparable. 
The  evaluation  design  must  provide  for  this  eventuality. 

Collection  of  data.  Achievement  data  should  be  collected  on  each 
trainee  at  the  end  of  AIT,  and  during  AIT  as  rel  ivant  data  are  available. 
After  approximately  two  months  of  unit  assign aent  following  AIT,  job 
performance  data  for  each  trainee  should  be  obta  ned  from  job  supervisors 
and  otherwise  as  dif cussed  in  Appendix  D.  In  addition,  trainee  performance 
on  the  Skill  Qualification  Test,  normally  take  i  six  months  after  unit 
assignment  will  pr  )vide  additional  data  re^  •  knowledge  and  skill 
levels,  and  some  aspects  of  job  performance. 
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User  accept.ince  data  should  be  obtained  from  trainees  immediately 
following  AIT,  and  again  from  them  and  from  job  supervisors  approximately 
two  months  following  unit  assignment.  Similar  data  should  be  obtained  from 
instructors  at  the  end  of  the  AIT  course  used  in  the  evaluations.  Also  at 
that  time,  the  information  discussed  above  regarding  characteristics  and 
utilization  of  training  devices  could  be  obtained  from  the  instructors. 

Analysis  of  data.  Most  criterion  data  can  be  analyzed  using  a  basic 
three-factor  analysis  of  variance  (ANOVA)  with  type  of  device,  instructional 
mode,  and  one  type  of  student  characteristic  comprising  the  three  factors. 
When  additional  data  for  other  factors  are  available,  such  as  a  second  type 
of  student  characteristic  or  location  of  training  rogram,  they  should  be 
added  to  the  basic  ANOVA. 

Another  kind  of  adaptation  to  the  basic  ANOVA  will  be  necessary  in  some 
comparisons  when  complete  sets  of  data  are  not  available  for  all  of  the 
three  basic  factors.  In  such  cases,  the  incomplete  factors  would  simply  be 
dropped  from  the  analysis. 

A  few  data  will  be  in  categorical  form  and  hence  not  in  a  metric 
suitable  for  ANOVA.  Chi  square,  or  some  other  nonparametric  statistical 
analysis  as  indicated,  would  then  be  used.  The  structure  of  the  analyses 
should  conform  to  the  factor  structure  of  the  basic  ANOVA,  although  no  more 
than  two  of  these  factors  would  be  involved  in  a  single  analysis.  Repeated 
analyses  of  the  same  data  with  various  factor  combinations  would  then  be 
necessary  to  provide  a  complete  picture  of  the  results. 

Support  Personnel 

Seven  types  of  technical  capabilities  must  be  represented  on  the  team 
that  conducts  the  training  evaluations.  These  capabilities  are: 

1.  Expertise  in  administering  training  program  evaluations; 

2.  Expertise  in  job/task  analyses  and  in  deriving  training 
requirements  from  them; 

3.  Expertise  in  training  course  design,  with  special  emphasis  on 
training  device  utilization; 

4.  Expertise  in  developing  measuring  instruments; 

5.  Expertise  in  experimental  design,  especially  the  adaptation  of 
design  requirements  to  varying  local  training  conditions; 

6.  Expertise  in  data  analysis  in  general,  and  in  ANOVA  and  chi  square 
analyses  in  particular;  and 
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Technical  expertise  in  subject  matter 


IV.  DISCUSSION 


Army  maintenance  training  is  undergoing  changes  that  parallel  many  of 
those  occurring  in  other  types  f  Army  training.  Many  of  these  changes  are 
quite  fundamental  in  nature,  especially  those  relating  to  the  use  of  devices 
to  support  that  training.  Some  of  these  changes — e.g.,  the  shift  toward 
performance-based,  individualized  instructional  programs — have  been 
disrussed  in  this  report,  as  have  their  implications  for  the  training  device 
coni  epts  developed.  Clearly,  the  emphasis  on  reducing  training  costs 
wherever  possible  is  going  to  continue,  but  this  cannot  be  done  at  the 
expense  of  the  resultant  maintenance  skills.  In  fact,  the  need  is  for  a 
training  system  that  can  produce  mechanics  with  both  increased  breadth  and 
depth  of  skills.  Thus,  the  demands  being  placed  on  the  Army  maintenance 
training  system  are  increasingly  severe. 

Over  and  above  these  continuing  considerations,  there  is  the  even  more 
critical  requirement  that  operational  units  be  combat  ready.  In  this 
regard,  the  undesirability  of  diverting  operational  vehicles  and  equipment 
from  their  intended  mission  use  to  institutional  and  unit  training  programs 
has  been  discussed.  Another  consideration  related  to  combat  readiness  is 
the  assumption  that  future  conflicts  will  not  allow  the  luxury  of  a  lengthy 
period  in  which  the  unit  can  build  its  operational  capabilities  through 
OJT.  1/  Units  must  be  ready  to  perform  operationally  without  delay. 

The  problems  that  this  state  of  affairs  presents  for  maintenance 
training  are  complex.  The  inventory  of  vehicles  and  equipment  the  mechanic 
must  be  prepared  to  maintain  is  extensive,  but  the  capability  of  present 
institutional  and  init  maintenance  training  programs  and  equipment  to 
provide  such  prepara  :ion  is  limited.  While  there  are  many  reasons  for  these 
training  program  limitations,  the  one  of  major  concern  here  is  the  heavy 
reliance  on  the  use  of  actual  equipment  to  support  training,  i.e.,  the  use 
of  AETs.  Aside  fron  their  sometimes  substantial  cost  and  their  generally 
poor  suitability  for  instructional  use,  AETs  present  a  further  diffic- Ity  in 
that  the  Army  cannot  realistically  expect  to  provide  them  for  all  of  its 
vehicles  and  equipment,  nor  can  it  expect  to  expose  each  trainee  to  all  AETs 
that  do  exist.  Thus,  other  solutions  warrant  examination. 

In  the  present  effort,  the  potential  of  RPF  devices  to  meet  many  of  the 
maintenance  training  needs  presently  being  met  through  use  of  AETs  has  been 
examined,  and  several  such  device  concepts  have  been  described.  The  RPF 
concepts  described  tT'eat  four  high-priority  training  areas,  areas  involving 
both  substantial  numbers  of  trainees  and  numbers  of  costly  AET  devices. 
Because  of  these  factors,  the  RPF  concepts  presented  offer  a  potential  for 
substantial  future  cost  savings.  In  addition,  they  offer  the  potential  for 
more  effective  train: ng  in  certain  skills  presently  trained  inadequately  or 
not  at  all  with  existing  AET  devices  and  for  training  for  a  much  broader 


—  Th  -S  assumption  has  lead  some  to  characterize  any  possible  future 
conflict  as  a  "come-as-you-are  war."  Perhaps  less  colorfully,  this 
assumption  is  reflected  in  the  emphasis  in  current  Army  doctrine  on  "winning 
the  first  battle." 
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spectrum  of  current  inventory  vehicles.  It  is  this  extended  training 
potential  'hat  will  ultimately  Impact  most  heavily  on  future  Army  unit 
readiness. 

In  terms  of  training  device  technology,  the  RPF  concepts  described 
offer  the  potential  for  development  of  multipurpose  families  of  devices  to 
support  maintenance  training  of  broad  scope.  The  concepts  could  be  adapt¬ 
able  to  either  institutional  or  unit  training  and,  thus,  provide  a  means  of 
enhancing  Army  maintenance  training  at  all  levels.  Ine  mechanic  who  has 
been  trained  on  a  particular  subset  of  vehicles  or  systems  and  is  then 
expected  to  effectively  maintain  a  vehicle  subset  that  may  be  quite 
different  faces  a  considerable  problem.  Hence,  the  flexibility  the 
multipurpose  device  presents  should  be  attractive  to  the  unit  commander  as 
well  as  to  the  institutional  training  manager. 

Obviously,  the  potential  benefits  described  will  be  realized  only  if 
the  device  concepts  are  carried  through  to  prototype  development  and  then 
subjected  to  evaluation  as  to  their  training  effectiveness  and  possible  cost 
benefits.  This  report  describes  the  major  features  of  such  evaluation.  It 
should  be  noted,  though,  that  the  training  effectiveness  evaluation  of  the 
various  device  concepts  need  not  cover  every  possible  implementation  of  the 
concept,  i.e.,  its  extension  to  every  potential  system  or  vehicle.  Concept 
feasibility  can  be  evaluated  on  a  more  restricted  scale.  If  results  are 
favorable,  then  the  extensions  can  be  made,  where  deemed  appropriate  by  the 
Army,  on  production  devices. 

In  pursuing  the  development  and  evaluation  of  prototype  RPF  devices  of 
the  types  described,  special  attention  must  be  paid  to  the  task-analytic 
basis  from  which  they  would  be  developed.  As  is  noted  elsewhere  in  this 
report,  the  task  listings  and  task  analyses  on  which  the  various  current 
school  training  courses  have  been  built  are  of  variable  nature.  It  is 
do  :btful  that  any  of  the  existing  task  lists  or  analyses  is  sufficient  for 
the  development  of  detailed  performance  specifications  for  the  RPF  devices. 
Therefore,  if  the  Army  pursues  any  of  the  RPF  device  concepts  described  here 
to  the  prototype  development  stage,  a  first  necessary  step  would  be  the 
conduct  of  a  thorough  task  analysis  for  the  training  of  concern  in  order 
that  the  characteristics  of  the  resultant  device  would  best  relate  to  the 
critical  skill  and  training  requirements.  To  omit  this  step  would  ilmost 
surely  lead  to  devices  that  were  not  optimal  for  the  actual  tialning 
requirements  of  concern.  This  step  is,  of  course,  a  highly  Important  step 
in  the  development  of  any  training  device,  whatever  its  level  of  complexity. 
However,  when  device  physical  fidelity  is  being  reduced  deliberately,  as  in 
the  RPF  device,  particular  care  must  be  given  to  Insure  that  critical 
task/psychological  aspects  of  training  fidelity  are  not  inadvertently 
reduced. 

The  Army  has  made  great  progress  in  the  use  of  complex  simulation 
devices  to  support  operator  training  in  aviation,  and  it  has  similar 
programs  in  development  for  tank  crew  training.  Maintenance  training 
devices  have  received  much  less  attention,  but  the  current:  Army  program  to 
develop  such  devices,  of  which  the  present  effort  is  a  part,  reflects  an 
Increased  overall  recognition  of  the  Importance  of  and  interest  in 
maintenance  training  in  the  Army.  The  large  numbers  of  hardware  systems  and 
trainees  Involved  in  maintenance  training  illctate  that  training  device 
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approaches  such  as  the  RPF  play  an  important  part  in  tliat 
will  not  be  the  only  device  approach  of  importance 
maintenance  training-complex  devices  and  AETs  will  have 
role — but  the  potential  it  represents  appears  considerable 
thorough  investigation  and  evaluation  by  the  Army, 
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APPENDIX  A 


ARMY  MAINTENANC,!:  TRAINING 


This  appendix  presents  information  developed  in  the  Task  1  survey 
pertinent  to  Amy  training  courses  in  the  automotive,  track  vehicle,  and 
heavy  equipment  maintenance  areas.  The  first  part  of  the  appendix  is  a 
discussion  of  the  manner  in  which  the  2.4  major  ai  eas  or  domains  of 
maintenance  training  were  developed  for  use  in  Task  2.  ?our  of  the  2  +  areas 
were  then  selected  for  the  development  of  RPF  concepts  in  Tasks  3  and  4. 
For  further  discussion  of  the  four  areas  selected,  see  Section  III  of  the 
basic  report  and  Appendix  C. 

The  second  pm.  of  this  appendix  presents  s(  le«.;ted  descriptive 
information  covtsring  the  12  MOS-producing  courses  covere  f  in  the  Task  1  Army 
training  survey. 

MAINTENANCE  TRAINING  CATEGORIES 

MOS  and  Training  Category 


The  study  was  initially  designed  to  focus  icparatcly  on  the  areas  of 
automotive,  track  vehicle,  and  heavy  equipment  miintenatce  training.  Data 
developed  during  the  Task  1  survey,  however,  indicated  the  desirability  of 
combining  certain  aspects  of  training  in  these  are  is.  For  example, 
automotive  repairmen  (MOSs  63H10  and  63H20)  and  heavy  equipment  me  hanics 
(MOSs  62B10  and  62B20)  receive  training  in  maintenance  of  both  wheeled  and 
tracked  vehicles.  Furthermore,  the  track  vehicle  mei  hanics  (MOS  63C10)  also 
receive  training  on  wheeled  vehicles.  In  contrast,  ti  rret  mechanics  and 
repairmen  (MOSs  45K10,  4bK20,  45Nl0j  45P10,  and  45R10)  ire  confronted  i i th 
problems  and  systems  unique  to  turrets  and  with  maintenance  tasks  somev  at 
different  from  those  experienced  by  repairmen  ir  the  other  MOSs.  The 
decision  was  made  (with  SAG  approval),  therefore,  to  focus  the  research 
effort  on  only  two  major  categories  of  maintenance  trailing,  regardless  of 
MOS.  These  divisions  were:  (1)  wheel  and  track  vehicle  maintenance,  and 
(2)  turret  maintenance.  This  change  in  focus  allowed  a  greater  continuity 
and  consistency  of  effort  in  the  project. 

Phases  of  training.  Within  the  category  of  wheel  and  track  vehicle 
maintenance,  it  was  found  that,  regardless  of  the  type  of  vehicle  involved 
or  the  MOS  for  which  the  trainee  was  being  trained,  three  general  phases  of 
training  were  common.  These  phases  were:  (1)  a  power  plant  phase,  which 
includes  gasoline,  diesel,  and  multifuel  engines;  (2)  a  power  train  phase, 
including  clutches,  transmissions,  etc.;  and  (3)  a  "systems"  phase,  which 
includes  the  electrical,  fuel,  brake,  and  other  systems. 

Within  the  category  of  turret  maintenance,  it  was  found  th  it  the 
training  focused  around  the  four  primary  types  of  systems  found  in  a  turret: 
armament,  fire  control,  stabilization,  and  traversing/elevating.  These 
systems  are  composed  of  electrical  and/or  hydraulic  components,  and  the 
study  of  electrical  and  hydraulic  systems  is  a  major  par::  of  turret 
maintenance  training. 
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Skill  levels.  Initial  project  planning  centered  around  investigating 
maintenance  training  at  both  introductory  and  advanced  skill  levels.  This 
distinction  was  discarded,  however,  in  favor  of  one  based  on  level  of  main¬ 
tenance.  For  example,  a  62B10  (Skill  Level  1)  mechanic  is  supervised  by  a 
62B20  (Skill  Level  2)  mechanic,  but  they  both  perform  organizational  main¬ 
tenance.  Likewise,  a  63H10  repairman  is  supervised  by  a  63H20  repairman, 
but  they  both  perform  DS/GS  maintenance.  It  was  concluded,  therefore,  that 
a  clearer  distinction  existed  between  training  content  and  training  events 
for  organizational  maintenance  and  that  for  DS/GS  maintenance  than  existed 
between  introductory  and  advanced  training.  For  these  reasons,  the 
organizational-DS/GS  paradigm  was  used  throughout  the  study.  The  two  levels 
of  maintenance  of  concern  here — Organizational  Maintenance  and  Direct 
Support/General  Support  (DS/GS)  Maintenance — are  defined  as  follows: 

Organizational  Maintenance.  Maintenance  at  this  level  is 
normally  performed  on  line  at  the  operational  unit  level  and 
consists  of  both  preventive  and  operational  type  maintenance 
tasks.  It  is  directed  toward  daily  readiness  checks,  diagnostic 
tests,  lubrication,  adjustments,  and  the  capability  for  isolating, 
removing  and  replacing  components  or  subascembl ies  with  a  minimum 
of  effort  or  delay  to  the  unit's  operational  mission. 

Direct  Support/General  Support  (DS/GS)  Maintenance. 
Maintenance  at  this  higher  echelon  or  level  is  performed  primarily 
on  equipment  that  has  been  removed  from  the  vehicle  because  of 
parts  failures.  The  primary  function  is  to  isolate  the  actual 
failed  part  of  any  component  or  subsystem,  repair  or  replace  the 
failed  part,  and  then  test  and  verify  that  the  removed  unit  is 
satisfactory  for  return  to  stock.  DS/GS  maintenance  units  provide 
support  to  the  combat  units  (e.g.,  to  a  tank  company),  but  are  not 
direct  combat  units  themselves. 

Task  types.  The  wide  variety  of  maintenance  skills  taught  in  the 
courses  surveyed  could  be  classified  into  three  general  types  of  tasks: 
motor,  cognitive,  and  troubleshooting.  These  task  types  utilized  here  are 
not  intended  as  a  definitive  tack  taxonomy  in  the  formal  sense.  Rather, 
they  are  intended  as  a  simple  means  of  categorizing  the  various  training 
objectives  in  a  form  that  is  manageable  with  reference  to  RPF  device 
definition.  The  task  types  are  defined  as  follows: 

Motor  Tasks.  Those  tasks  involving  primarily  physical 
manipulation  of  tools  and/or  equipment.  Included  in  this  category 
are  such  activities  as  removal  and  replacement  of  vehicle  parts, 
assembly  and  disassembly  of  cot^ponents,  cleaning  of  parts,  and 
loosening  or  tightening  of  nuts,  bolts,  screws,  etc. 

Cognitive  Tasks.  Those  tasks  requiring  primarily  that  the 
learner  be  in  command  of  a  particular  body  of  cognitive  knowledge. 
Included  in  this  category  are  such  activities  as  identification  or 
recall  of  names,  locations,  facts,  numbers,  procedures,  rules,  and 
symbols.  The  possession  of  such  information  is  an  internal. 
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covert  activity,  and  an  external  action  or  task  (e.g.,  responding 
to  a  test  question  or  pointing  to  an  engine  component)  may  be 
required  to  demonstrate  the  learner's  level  of  cognitive 
knowledge. 

Troubleshooting  Tasks.  These  tasks  require  learners  to 
combine  both  motor  tasks  and  cognitive  knowledge  of  a  mechanical, 
hydraulic,  or  electrical  system  or  component  in  the  analysis  of 
trouble  symptoms  and  identification  and  repair  of  the  malfunction 
indicated  by  the  symptoms.  This  category  of  tasks  can  involve 
performance  of  various  tests  and  calibrations,  and  requires  skills 
in  decision  making.  The  learner  is  thus  required  to  function  at 
the  higher  cognitive  levels  of  analysis,  synthesis,  and  evaluation 
in  order  to  perform  these  tasks. 

The  proportion  of  the  tasks  taught  falling  into  each  of  these  task 
types  was  determined  by  analyzing  the  objectives  of  the  various  courses  as 
they  were  stated  in  the  POIs.  This  analysis  provided  an  indication  of  the 
relative  emphasis  placed  on  the  various  types  of  tasks  trained  in  the 
institutional  setting.  Table  A-1  shows  the  classification  of  task  type  for 
the  various  training  objectives  covered  in  the  several  courses  as  a  function 
of  category  of  training  and  level  of  maintenance.  As  can  be  seen, 
troubleshooting  is  a  relatively  important  factor  in  all  category-level 
combinations . 


Table  A-1 


Objectives  by  Task-Category-Level 


MAINTENANCE 

CATEGORY  &  LEVEL 

Motor 

TASK  TYPE 
Cognitive 

Trouble¬ 

shooting 

Wheel  &  Track  Vehicle; 
Organizational  Level 

56  £/ 

1 

43 

Wheel  &  Track  Vehicle; 

DS/GS  Level 

32 

32 

36 

Turret ; 

Organizational  Level 

49 

5 

46 

Turret ; 

DS/GS  Level 

58 

5 

37 

£./Numbers  in  table  refer  to  percentage  of  training  objectives 
taught  falling  in  each  maintenance  category-level  combination  over 
all  associated  courses  of  instruction. 
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Areas  of  maintenance  training.  As  a  result  of  combining  the  two  cate¬ 
gories  of  maintenance  (wheel /track  vehicle  and  turret),  the  two  levels  of 
maintenance  (organizational  and  DS/GS),  the  three  general  training  phases 
common  to  the  wheel/track  vehicle  maintenance  courses  (power  plant,  power 
train,  and  systems),  and  the  three  task  types  (motor,  cognitive,  and 
troubleshooting),  some  24  major  areas  of  maintenance  training  were 
identified  for  possible  RPF  device  development.  Of  these,  18  concerned 
wheel/track  vehicle  maintenance,  while  six  pertained  to  turret  maintenance. 
These  are  shown  in  Tables  A-2  and  A-3,  respectively. 


Table  A-2. 

Areas  of  Wheel  and  Track  Vehicle  Maintenance  Training 

TASK  TYPE 


MAINTENANCE 

LEVEL 

TRAINING 

PHASE 

Motor 

Cognitive 

Trouble¬ 

shooting 

Power  Plant 

Area 

1 

/  rea 

2 

Area  3 

Organizational 

Power  Train 

Area 

4 

/rea 

5 

Area  6 

Systems 

Area 

7 

Area 

8 

Area  9 

Power  Plant 

Area 

10 

Area 

11 

Area  12 

DS/GS 

Power  Train 

Area 

13 

Area 

14 

Area  15 

Systems 

Area 

16 

Area 

17 

Area  18 

Table 

A-3. 

Areas  of  Turret  Maintenance  Training 

MAINTENANCE 

LEVEL 

Motor 

TASK  TYPE 
Cognitive 

Trouble¬ 

shooting 

Organizational 

Area  19 

Area 

20 

Area  21 

DS/GS 

Area  22 

Area 

23 

Area  24 

These  24  areas  were  rated  (see  Appendix  C)  to  establish  RPF 
developmental  priorities  for  Tasks  3  and  4.  Eventually,  the  following  four 
subareas  were  chosen  as  the  aspects  of  training  in  whidi  to  concentrate 
activities  in  Tasks  3  and  4;  (1)  troubleshooting  engines  and  related 
systems  at  the  organizational  level;  (2)  troubleshooting  track  vehicle 
track/suspension  systems  at  the  organizational  level;  (3) 
removal/replacement  of  engines  and  power  pecks  at  the  DS/GS  level;  and  (4) 
troubleshooting  turret  electrical  and  hydraulic  systems  at  both 
organizational  and  DS/CS  levels. 


DESCRIPTIONS  OF  COURSES  SURVEYED 

4  *  Twelve  MOS-producing  Army  maintenance  courses  were  covered  in  the  Task 

^  1  survey.  This  section  of  this  appendix  presents  selected  descriptive 

information  about  these  courses.  For  each  course  the  following  items  are 
given:  Job  (MOS)  title;  MOS  number;  training  location;  annual  student  load; 
purpose  of  course;  mode  of  instruction;  course  length;  and  AETs  used  in  the 
course. 


1.  Title :  Construction  Equipment  Mechanic  (Skill  Level  1) 


^  ■ 

MOS:  62B10 

Training  Location:  Fort  Leonard  Wood,  MO 

Annual  Student  Load:  2000 

Purpose:  To  train  enlisted  personnel  in  the  skills  and  knowledge 

necessary  to  perform  organizational  maintenance  on  wheeled  and 
track  construction  equipment  at  skill  level  one  of  MOS  62B. 

Mode:  Lock-Step 

Length:  6  weeks 

AETs  Used:  V4  ton,  21/2  ton  and  5  ton  trucks 

Cranes 

Dozers 

Rollers 

Graders 

Air  compressors 

Also  components  from  these  end  items. 

are  unserviceable  vehicles. 

Air  compressors  used  in  three  courses 
Warrant  Officer  Technician) 

All  end  items 

(62B10,  62B20, 

2.  Title:  Construction  Equipment  Mechanic  (Skill  Level  2) 

MOS:  62B20 

Training  Location:  Fort  Leonard  Wood,  MO 

Annual  Student  Load:  250 

V 

■f 

Purpose:  To  train  enlisted  personnel  in  the  skills  and  knowledge 

necessary  to  perform  organizational  maintenance  on  wheeled  and 
track  construction  equipment  at  skill  level  two  of  MOS  62B. 

Mode:  Lock-step 

Length:  5  weeks 

^  ■ 

AETs  Used:  Components  only,  taken  from  all  types 
equipment . 

of  construction 

53 

^  p- 


3.  Title:  Wheel  Vehicle  Mechanic 


MOS:  63B10 


Training  Location:  Fort  Leonard  Wood,  MO 
Annual  Student  Load:  2000 

Purpose :  To  train  enlisted  personnel  in  the  skills  and  knowledge 

necessary  to  perform  organizational  maintenance  on  automotive 
wheeled  vehicles  at  skill  level  one  of  MOS  63B. 

Mode:  Self-paced 

Length :  8  weeks 

AETs  Used;  Va  ton,  21/2  ton  and  5  ton  trucks 
Crane  components 
Dodge  Power  Wagons 
Gama  Goat 

4.  Title;  Track  Vehicle  Mechanic 
MOS:  63C10 


Training  Location:  Ft.  Knox,  KY 
Annual  Student  Load:  6000 

Purpose ;  To  train  enlisted  personnel  in  the  skills  and  knowledge 
necessary  to  perform  organizational  maintenance  on  wheeled  and  track 
tank-automotive  vehicles  at  skill  level  one  of  MOS  63C. 

Mode :  Lock-step 

Length :  13  weeks 

AETs  Used:  Wheeled  phase  r 

Va  ton,  21/2  ton  and  5  ton  trucks 

Gama  Goats 

Goers 

Engines  in  test  cells 
Tracked  phase 

M109  self-propelled  howitzers 
MllO  self-propelled  howitzers 
Ml 13  armor,  d  personnel  carriers 
M60  tanks 

M60A1 ,  M60A2,  and  M551  tanks 
M88  vehicle  track  recovery  (heavy) 

M578  vehicle  track  recovery  (light' 

Engines  in  test  cells 


5A 


5.  Title: 


Fuel  and  Electrical  Systems  Repairman 


MOS:  63G10 

Training  Location:  Aberdeen  Proving  Ground,  MD 
Annual  Student  Load:  400 

Purpose :  To  train  enlisted  personnel  in  the  skills  and  knowledge 

necessary  to  perform  direct  and  general  support  maintenance  on  fuel 
and  electrical  system  components  and  brake  systems.  This  course 
trains  both  skill  levels  one  and  two  of  MOS  63G. 

Mode :  Self-paced 

Length :  12  weeks  - 

AETs  Used:  Electrical  and  fuel  systems  components 
Gasoline  and  diesel  engines 
1/4  ton  trucks  (lights,  wiring,  and  brakes) 

6.  Title ;  Automotive  Repairman  (Skill  Level  1) 

MOS:  63H10 

Training  Location:  Aberdeen  Proving  Ground,  MD 
Annual  Student  Load:  2500 

Purpose :  To  train  enlisted  personnel  in  the  skills  and  knowledge 
necessary  to  perform  direct  and  general  support  maintenance  on 
tank-automotive  wheeled  and  tracked  vehicles  at  skill  level  one  of 
MOS  63H, 

Mode :  Self-paced 

Length :  10  weeks 

AETs  Used:  Basic  knowledge  -  1/4  ton  trucks 

Wheeled  phase  -  21/2  ton  and  5  ton  trucks 

Tracked  phase  -  M60A1 ,  M60A2,  M551  tanks 

Self-propelled  howitzers 
Special  purpose  tracked  vehicles 
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7,  Title :  Automotive  Repairman  (Skill  Level  2) 
MOS:  63H20 


Training  Location:  Aberdeen  Proving  Ground,  MD 
Annua  Student  Load:  250 

Purpose :  To  train  enlisted  personnel  in  the  skills  and  knowledge 

necessary  to  perform  direct  and  general  support  maintenance  on 
tank-automotive  wheeled  and  tracked  vehicles  at  skill  level  two  of 
MOS  63H. 

Mode :  Lock-step 

Length :  11  weeks 

AETs  Used:  No  end-item  vehicles 

All  bench-item  components  from  wheeled  and  tracked  vehicles 

8.  Title:  Tank  Turret  Repairman  (Skill  Level  1) 

MOS:  45K10 


Trailing  Location:  Aberdeen  Proving  Ground,  MD 
Annual  Student  Load:  600 

Purpose :  To  train  enlisted  personnel  in  the  skills  and  knowledge 
necessary  to  perform  direct  and  general  support  maintenance  on  tank 
turrets,  armored  reconnaissance  vehicles,  and  armored  personnel 
carrier  cupolas  at  skill  level  one  of  MOS  45K. 

Mode :  Lock-step 

Length :  8  weeks 

AETs  Used;  M60A1  tank  turrets 
M60A2  tank  turrets 
M551  tank  turrets  and  gun  tubes 
M551  tank 
M27  cupolas 


^  ► 


9.  Title :  Tank  Turret  Repairman  (Skill  Level  2) 

MOS:  45K20 

Training  Location:  Aberdeen  Proving  Ground,  MD 
Annual  Student  Load:  15 

Purpose :  To  train  enlisted  personnel  in  the  skills  and  knowledge 

necessary  to  perform  direct  and  general  support  maintenance  on  tank 
turrets,  armored  reconnaissance  vehicles,  and  armored  personnel 
carrier  cupolas  at  skill  level  two  of  MOS  45K. 

Mode :  Lock-step 

1 

Length :  16  weeks 

AETs  Used:  M60A1,  M60A2,  and  M551  tank  turrets  and  gun  tubes 
Title :  Tank  Turret  Mechanic 
MOS:  45N10 

Training  Location:  Ft.  Knox,  KY 
Annual  Student  Load;  300 

Purpose :  To  train  enlisted  personnel  in  the  skills  and  knowledge 
necessary  to  perform  organizational  maintenance  on  tank  turret 
systems  at  skill  level  one  of  MOS  45N. 

Mode :  Lock-step 

Length :  81/2  weeks 

AETs  Used:  M60A1  tank  turrets 

M60A1  turret  trainers  (cut-away  turrets) 

11.  Title :  Sheridan  Turret  Mechanic 

MOS:  45P10 

Training  Location:  Ft.  Knox,  KY 
Annual  Student  Load:  200 

Purpose :  To  train  enlisted  personnel  in  the  skills  and  knowledge 

necessary  to  perform  organizational  maintenance  on  M551  series  AR/AAV 
turret  systems  at  skill  level  one  of  MOS  45P. 


Mode :  Lock-step 

Length :  81/2  weeks 

AETs  Used:  M551  tank  turrets 
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12.  Title:  Missile  Tank  Turret  Mechanic 


MOS:  45R10 

Training  Location:  Ft.  Knox,  KY 
Annual  Student  Load:  25 

Purpose :  To  train  enlisted  personnel  in  the  skills  and  knowledge 

necessary  to  perform  organizational  maintenance  on  M60A2  tank 
turret  systems  at  skill  level  one  of  MOS  45R. 

Mode :  Lock-step 

Length :  IIV2  weeks 

AETa  Used :  M60A2  tank  turrets 

M60A2  turret  trainers  (cut-away  turrets) 
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APPENDIX  B 


SURVEY  PROCEDURE 

This  appendix  describes  the  general  procedure  used  in  the  field  survey 
of  selected  Army  and  non-Army  maintenance  training  sites  that  comprised  Task 
1.  The  principal  purpose  of  the  survey  was  to  identify  current  maintenance 
training  practices,  including  the  devices  used  to  support  that  training,  in 
order  to  provide  the  information  required  for  subsequent  tasks.  More 
specifically,  the  survey  information  was  necessary  to  the  selection  of 
training  areas  in  which  possible  development  of  RPF  alternatives  to  AET 
devices  would  be  explored. 

The  first  section  of  this  appendix  describes  the  general  survey 
procedure  and  lists  the  places  and  persons  surveyed.  The  second  section 
presents  the  interview  guide  used  in  the  survey,  while  the  third  section 
presents  certain  general  observations  related  to  the  survey  and  resulting 
data.  Further  information  concerning  the  specific  Army  training  courses 
examined  in  the  survey  is  contained  in  Appendix  A. 

PROCEDURE 

General  Procedure 


The  general  procedure  followed  in  the  survey  was  to  conduct  a  series  of 
interviews  with  the  personnel  responsible  for  the  management  of  the 
maintenance  training  courses  of  concern  and,  wherever  possible,  with  the 
personnel  actually  administering  the  instruction.  When  time  and 
circumstances  permitted,  the  survey  team  observed  first-hand  the 
instructional  segments  in  which  the  AET  devices  were  being  used.  Interviews 
were  administered  using  a  structured  interview  guide.  Advance  coordination 
with  interviewees  was  effected  through  telephonic  contact  and  the 
transmission  of  official  messages. 

As  can  be  seen  from  review  of  the  interview  guide  in  the  next  section 
of  this  appendix,  the  major  areas  of  concern  in  the  interview  were  as 
follows : 

•  Types  of  AETs  and  the  extent  of  their  use 

•  Skills/tasks  taught  with  the  AETs 

•  Course  information 

•  Trainee  information 

The  time  required  to  conduct  the  interviews  varied  depending  upon  the 
context  of  the  interview.  Persons  in  authority  such  as  division  chiefs  and 
training  directors  were  asked  to  respond  to  the  entire  guide,  and  this 
generally  consumed  about  one  hour.  When  observing  the  instruction  taking 
place  in  the  shops,  only  those  items  dealing  with  AET  usage,  skills/tasks 
taught,  method  of  measurement  and  problem  areas  were  asked  of  the 
instructors.  These  items  consumed  as  little  as  one  or  two  minutes  and  as 
much  as  20  minutes,  depending  on  how  much  time  the  instructor  had  available 
and  how  much  detail  he  provided. 
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Army  Courses  Surveyed 

Three  major  Army  training  centers  were  visited  and  12  courses  were 
surveyed.  Listed  below  are  the  three  centers  and  the  courses  surveyed  at 
each,  identified  by  MOS  number  and  name. 

1.  U.S.  ARMY  ORDNANCE  AND  CHEMICAL  CENTER  AND  SCHOOL 

ABERDEEN  PROVING  GROUND,  MARYLAND 

•  MOS  63H10  Automotive  Repairman  (Skill  Level  1) 

•  MOS  63H20  Automotive  Repairman  (Skill  Level  2) 

•  MOS  63G10  Fuel  and  Electrical  Systems  Repairman 

•  MOS  A5K10  Tank  Turret  Repairman  (Skill  Level  1) 

•  MOS  45K20  Tank  Turret  Repairman  (Skill  Level  2) 

2.  U.S.  ARMY  ARMOR  SCHOOL 

FORT  KNOX,  KENTUCKY 

•  MOS  45N10  Tank  Turret  Mechanic 

•  MOS  45P10  Sheridan  Turret  Mechanic 

•  MOS  45R10  Missile  Tank  Turret  Mechanic 

•  MOS  63C10  Track  Vehicle  Mechanic 

3.  U.S.  ARMY  TRAINING  CENTER 

FORT  LEONARD  WOOD,  MISSOURI 

•  MOS  62B10  Engineer  Equipment  Mechanic  (Skill  Level  1) 

•  MOS  62B20  Engineer  Equipment  Mechanic  (Skill  Level  2) 

•  MOS  63B10  ^Theel  Vehicle  Mechanic 

The  following  personnel  were  contacted  during  the  survey: 

1.  U.S.  ARMY  ORDNANCE  AND  CHEMICAL  CENTER  AND  SCHOOL  (ABERDEEN) 

•  Training  Development  -  Mr.  Seese  &  Miss  Gaventer 

•  TASO  -  Mr.  Dees 


•  63H10  -  MAJ  Franks,  Division  Chief 

SGT  Tukums ,  Branch  Chief 

SGT  Jones  &  SGT  Kelly,  Senior  Instructors 

•  63H20  -  SGT  Doss,  Senior  Instructor 
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63G10 


•  63G10  -  CPT  Murphy,  Division  Chief 

CW4  Brady,  Branch  Chief 

•  45K10  -  CPT  Dyson,  Branch  Chief 

SGT  Owens,  Senior  Instructor 

e  45K20  -  SGT  Chappel  (USMC) ,  &  SGT  Langley,  Senior  Instructors 
U.S.  ARMY  ARMOR  SCHOOL  (FORT  KNOX) 

•  45N10  -  Mr.  Manning,  Branch  Chief 

•  45P10  -  Mr.  Pierce,  Branch  Chief 

•  45R10  -  Mr.  Szczapinski,  Branch  Chief 


63C10  -  MAJ  Lynn,  Division  Chief 

MAJ  Randles,  Operations  Officer 

Mr.  Bateman,  Engine  and  Electric  Division 


U.S.  ARMY  TRAINING  CENTER  (FORT  LEONARD  WOOD) 


•  CAPT  Trien,  Training  Command 

Mr.  Edmondson,  Training  Development 

•  62B10  -  LT  Craig,  Company  Commander 

CW3  Viau,  Course  Chief 

•  62B20  -  CW3  Staton,  Course  Chief 

•  63B10  -  LT  Allaben,  Officer  in  Charge 

SGT  Reigle  &  SGT  South,  Senior  Instructors 

Non-Army  Courses  Surveyed 

A  corollary  effort  of  Task  1  was  an  investigation  of  maintenance 
training  practices  of  selected  commercial  and  non-Army  military  agencies. 
Maintenance  training  at  several  such  sites  was  observed  to  determine  if  RPF 
alternatives  already  existed  in  the  areas  of  training  where  the  Army  is 
currently  using  AETs.  The  purpose  of  this  effort  was  to  establish  a  basis 
for  com^'aring  Army  practices  with  the  practices  of  non-Army  agencies  engaged 
in  simj  r  areas  of  maintenance  training.  The  non-Army  agencies  and 
specific  areas  of  training  surveyed  were: 

1.  U.S.  AIR  FORCE 

CHANUTE  AIR  FORCE  BASE,  RANTOUL,  ILLINOIS 

•  Base  Vehicle  Mechanic 

•  Special  Vehicle  Mechanic 


2.  U.S.  NAVY 

NAVAL  CONSTRUCTION  TRAINING  CENTER,  GULFPORT,  MISSISSIPPI 


•  Construction  Mechanic,  A,  J,  and  C  Schools 


3.  MOTECH  AUTOMOTIVE  EDUCATION  CENTER 

LIVONIA,  MICHIGAN 

•  Automotive  Service  and  Body  Repair 

4.  JOHN  DEERE  CENTRAL  DIVISION 

MEMPHIS,  TENNESSEE 

•  Heavy  Equipment  Maintenance 

5.  ATLANTA  AREA  TECHNICAL  SCHOOL 

ATLANTA,  GEORGIA 

•  Automotive  Mechanic 

Personnel  contacted  at  these  non-Army  sites  included: 

1 .  CH/ NUTE  AFB 

•  COL  Yeabower,  School  Commander 

•  Mr.  Carter,  Civilian  Advisor  to  Commander 

•  Mr.  Harper,  Training  Department 

•  SGT  Patterson  &  SGT  Bloomer,  Instructors 

2.  NAVAL  CONSTRUCTION  TRAINING  CENTER 

•  CDR  Griffith,  Commanding  Officer 

•  CW03  Hays,  Equipment  Schools  Department  Head 

•  Chief  Seiler,  Senior  Instructor 

•  Mr.  Matthews,  Training  Department 

3.  MOTECH  AUTOMOTIVE  EDUCATION  CENTER 

•  Mr.  Vorasco,  Chief  Instructor 

•  Mr.  Haight,  Instructional  Developer 

4.  JOHN  DEERE  CENTRAL  DIVISION 

•  Mr.  Love,  Service  Manager 

•  Mr.  Phillips,  Instructor/Instructional  Developer 

5.  ATLANTA  AREA  TECHNICAL  SCHOOL 

•  Mr.  Friedman,  Instructional  Specialist 
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INTERVIEW  GUIDE 


1.  Training  Location 
2^  Name/Rank/Position  of  Respondent 

3.  Course 

4.  Method  of  Instruction  (Lock-Step  or  Self-Paced) 

5.  Annual  Trainee  Flow 

6.  Trainee  Characteristics 

Age 

Education  level 
Reading  level 
Time  in  Army 
Attrition  rate 

7.  Availability  of: 

Task  List 
POI 

Soldiers  Manual 
SQT 

8.  How  trainee  performance  is  measured 

Written  tests 
Performance  exams 

9.  Trainee  Movement  Through  Course 

Sequence 

Time 

10.  Significant  Problem  Areas  or  Areas  Difficult  to 
Teach/Reasons 

Equipment  constraints 
Lack  of  equipDient 

Equipment  not  suitable  for  training 

Time  constraints 

Trainee  flow 

Task  complexity 

Subject-matter  difficulty 

Trainees'  lack  of  ability 

11.  Percent  of  Instruction  that  is  Hands-On 

Classroom  instruction 
Shop  instruction 

12.  AETs  Used  in  Course/Cost/Origin 

Classroom 

Shop 

13.  Skills/Tasks  Taught  with  AETs 

Cognit ive 

Identification  of  parts 
Location  of  parts 
Theory  of  operation 
Sequences /procedure 8 
Motor 

Removal /re placement 
Adjustment 
Troub leshoot ing 
Testing 

Problem  identification 
Fault  isolation 

Prescription  of  corrective  procedures 


OBSERVATIONS 

In  the  conduct  of  the  Task  1  survey  specific  data  were  gathered  and  a 
variety  of  less  formal  observations  were  made.  In  general,  the  nature  of 
the  maintenance  training  observed  can  be  described  as  (1)  lock-step,  (2) 
emphasizing  system  theory,  and  (3)  placing  heavy  reliance  on  the  use  of 
AETs.  Tliere  were  some  exceptions,  of  course,  but  the  support  for  the  use  of 
actual  equipment  in  training  was  virtually  unanimous.  Also,  there  was 
little  or  no  difference  of  consequence  between  the  Army  and  non-Army 
maintenance  training  courses  observed. 

The  following  statements  briefly  treat  several  of  the  areas  of  concern 
in  the  survey.  It  should  be  noted  that  these  statements  refer  only  to  the 
Army  training  courses. 

Task  and  Skills  Analysis  Basis 


The  investigators  in  this  study  were  asked  to  determine  the  task  and 
skills  analysis  basis  for  the  maintenance  training  programs  examined, 
particularly  the  relationship  of  training  to  the  critical  mainterance  tasks 
represented  in  Skill  Qualification  Tests  (SQTs).  The  interest  was  the 
extent  to  which  these  skills  would  be  reflected  in  the  intended  uses  of  AET 
devices  examined.  It  was  found  that  at  the  time  of  the  survey  no  SQTs  had 
been  developed  for  the  12  MOSs  st-veyed  and  that  Soldier's  Manuals  existed 
for  only  six  of  those  MOSs.  New  POIs  related  to  the  critical  task  list  for 
any  particular  MOS  had  not  yet  been  developed,  and  the  POIs  in  use  were 
based  on  the  TRADOC  systems  engineering  effort  of  several  years  ago.  As  a 
consequence,  none  of  the  ioli,  even  in  the  courses  where  Solider's  Manuals 
were  available,  was  directly  related  to  a  critical  task  list  or  a  Soldier's 
Manual.  Therefore,  no  particular  correlation  was  observed  between  critical 
task  lists  a'd  the  intended  uses  of  the  AET  devices  examined. 

Extent  of  Usage  of  AETs 

Approximately  90%  of  the  maintenance  training  observed  was  conducted 
with  actual  end-item  equipment  as  the  primary  training  device,  with 
virtually  all  of  that  90%  involving  the  teaching  of  motor  and 
troubleshooting  tasks.  This  extent  of  usage  of  AETs  was  common  in  both 
organizational  and  DS/GS  training.  The  remainder  of  the  instruction, 
particularly  the  teaching  of  cognitive  tasks,  was  conducted  using  a  variety 
of  audio-visual  media  and/or  panel  boards  constructed  with  actual  components 
from  vehicle  systems,  most  of  which  were  locally  fabricated  for  a  specific 
course.  Few  commercially  produced  training  aids  were  observed,  and  they 
were  usually  manufacturer-built  cutaway  engines  or  other  vehicle  components 
which  also  are  technically  defined  as  AETs. 

Intended  and  Actual  Usage  of  AETs 

Approximately  75%  of  the  AETs  used  in  the  courses  observed  were,  due  to 
age  and/or  damage,  items  of  equipment  not  usable  in  the  units  and  dedicated 
solely  to  maintenance  training.  Once  the  equipment  is  found  to  be 
nonfunctional  for  unit  use  and  dedicated  to  maintenance  training,  it  is 
rarely  used  for  any  other  purpose.  The  investigator:  in  this  study  found  no 
great  discrepancy  between  intended  and  actual  uses  of  AETs  used  for 
maintenance  training. 
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Multiple  Uses  of  AETs 


In  the  maintenance  training  observed,  the  trainees  moved  from  shop  to 
shop  as  they  progressed  through  their  training.  The  AETs  used  in  each  phase 
were  usually  dedicated  to  that  phase,  and  use  of  an  individual  AET  for  more 
than  one  phase  of  training  was  rare.  An  exception  was  found  in  the  MOS 
62B10/20  Engineer  Equipment  Mechanic  courses  at  Fort  Leonard  Wood.  The  air 
compressors  used  in  that  phase  of  training  were  also  used  in  Warrant 
Officer's  courses  that  are  taught  in  the  same  facilities. 

It  was  found,  however,  that  multiple  uses  of  classes  of  AETs  (as 
opposed  to  individual  items  of  equipment)  was  a  common  practice.  An  example 
is  the  M151  quarter-ton  truck,  commonly  known  as  the  Jeep.  This  vehicle  is 
readily  available  in  very  large  numbers  and  is  used  for  basic  instruction  in 
a  wide  variety  of  task  areas,  particularly  in  troubleshooting  and  repair  of 
fuel,  electrical,  and  brake  systems.  Multiple  use  of  other  classes  of 
vehicles,  especially  the  various  weight  levels  of  large  trucks,  apparently 
is  also  widespread  throughout  Army  maintenance  training. 


APPENDIX  C 

SELECTION  OF  HIGH-INTEREST  TRAINING  AREAS 

This  appendix  describes  Che  manner  in  which  specific  areas  of 
maintenance  training  were  selected  for  the  development  of  possible  RPF 
devices.  This  selection  process  was  the  focus  of  Task  2  of  the  overall 
effort. 

As  described  in  the  basic  report  and  in  Appendix  A,  there  were  some  24 
areas  or  domains  of  maintenance  training  resulting  from  the  various 
combinations  of  (a)  training  category,  (b)  maintenance  level,  (c) 
instructional  phase,  and  (d)  task  type.  The  goal  of  Task  2  was  to  identify 
areas  among  these  24  that  would  be  of  highest  interest  to  the  Army  with 
reference  to  future  development  of  RPF  alternatives  to  AETs. 

Two  types  of  activities  were  involved  in  Task  2.  First,  a  method  for 
rating  the  24  instructional  areas  was  devised  and  applied  in  order  to  allow 
establishment  of  a  priority  ordering.  Second,  based  on  these  ratings,  four 
of  the  24  areas  were  selected  and  recommended  to  the  SAG  as  the  areas  in 
which  to  pursue  RPF  device  conceptual  development  in  Tasks  3  and  4. 

Six  dimensions  were  identified  as  especially  pertinent  to  the  selection 
of  areas  for  RPF  device  development.  These  were:  (1)  skill  complexity;  (2) 
subject-matter  difficulty;  (3)  commonality  to  other  courses;  (4)  trainee 
volume;  (5)  AET  effectiveness  as  a  teaching  aid;  and  (6)  AET  cost.  These 
dimensions  were  defined  as  follows: 

A.  Skill  complexity.  Factors  considered  here  were  the  degree  of 

complexity  required  in  the  manipulation  of  tools  and  equipment, 
the  sophistication  of  those  tools  and  equipment,  and  the  amount 
and  nature  of  the  cognitive  knowledge  required  to  perform  the 
skill. 

B.  Subject-matter  difficulty.  Considered  here  were  the  ability  level 
of  the  trainees,  the  degree  to  which  the  skill  or  knowledge  is 
abstract  rather  than  concrete,  the  complexity  of  the  interrelation¬ 
ships  between  equipment  systems  and/or  subsystems,  and  the 
reading  level  of  course  materials  or  manuals. 

C.  Commonality  to  other  courses.  The  primary  element  in  this 

dimension  was  the  percentage  of  the  courses  observed  in  which  a 
particular  skill  was  taught.  If  the  skill  was  taught  in  80%  or 
more  of  the  courses,  it  was  rated  Very  High  on  commonality.  It  was 
rated  High  if  taught  in  60-79%  of  the  courses;  Moderate  if  taught 
in  40-59%  of  the  courses;  Low  if  taught  in  20-39%  of  the  courses; 
and  Very  Low  if  taught  in  less  than  20%  of  the  courses  observed. 

D.  Trainee  volume.  The  annual  volume  of  trainees  flowing  through  a 

course  was  the  determining  factor  here.  The  ratings  were  as 

follows:  Very  High  ■  5,000  or  more  trainees  per  year;  High  « 

3,000-4,999  trainees  per  year;  Moderate  ■*  1,000-2,999  trainees  per 
year;  Low  ■=  500-999  trainees  per  year;  and  Very  Low  “  less  than 
500  trainees  per  year. 
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E.  Effectiveness  of  AET  as  a  teaching  aid.  Considerations  included 
the  aft's  ability  to  display the system  being  studied,  noise 
level,  safety,  ability  to  demonstrate  a  range  of  malfunctions, 
ease  of  insertion  of  malfunctions,  ease  of  use  for  multiple  tasks, 
ability  to  withstand  heavy  use,  adaptability  to  engineering 
changes,  ability  to  measure  trainee  performance,  ability  to  provide 
feedback  to  the  trainee,  and  presence  of  features  specifically 
designed  to  enhance  training. 

F.  AET  cost.  In  this  category.  Very  High  =  $100,000  and  over;  High  = 
$50,000-$99,999;  Moderate  =  $10,000-$49,999;  ^  =  $5,000-$9,999; 
Very  Low  =  leas  than  $5,000. 

Based  on  the  interview  data  developed  in  Task  1,  the  on-site 
observations  of  the  various  AET  devices  and  their  use  in  training,  and  the 
variety  of  other  material^/  gathered  relevant  to  Army  maintenance  and 
training,  the  research  team  assigned  five-point  rating  scale  values  to  each 
of  the  six  dimensions  identified  for  each  of  the  24  training  domains.  The 
five-point  scale  was  as  follows:  1  =  Very  High;  2  =  High;  3  *  Moderate;  4  = 
Low;  and  5  =  Very  Low.  In  applying  this  rating  scale  to  the  six  dimensions, 
a  degree  of  subjectivity  was  necessarily  present  on  some  dimensions,  while 
others  were  relatively  objective. 

The  results  of  this  rating  procedure  for  wheel  and  track  vehicle 
maintenance  training  are  shown  in  Table  C-1 ,  while  those  for  turret 
maintenance  training  are  given  in  Table  C-2.  In  developing  these  ratings, 
the  research  staff  discussed  each  factor  and  area  separately  in  light  of  the 
survey  data  and  experience  and  then  arrived  at  a  group  consensus  rating. 
Thus,  the  numbers  shown  in  Tables  C-1  and  C-2  represent  the  best  collective 
judgments  of  the  research  team  resulting  from  these  discussions. 

Based  on  the  ratings  and  other  pertinent  considerations,  the  research 
team  recommended  to  the  SAG  that  RPF  device  development  be  focused  on 
instructional  segments  falling  within  four  of  the  24  instructional  domains. 
The  four  areas  were:  (1)  troubleshooting  tasks,  power  plants, 

organizational  level  wheel  and  track  vehicle  mechanic;  (2)  troubleshooting 
tasks,  power  train,  organizational  level  wheel  and  track  vehicle  mechanics; 
(3)  motor  tasks,  power  plant,  DS/GS  level  wheel  and  track  vehicle  mechanics; 
and  (4)  troubleshooting  tasks,  both  organizational  and  DS/GS  level  turret 
mechanics.  ITie  particular  instructional  segments  ultimately  selected  for 
Tasks  3  and  4  were  taken  from  these  four  instructional  areas.  More  specifi¬ 
cally,  the  instructional  segments  selected  in  each  of  these  four  areas  were 
as  follows: 

i./Such  material  included  course  syllabi  and  programs  of  instruction, 
lesson  plans,  task  lists.  Soldier's  Manuals,  Skill  Qualification  Tests, 
device  costs,  student  flow,  and  similar  training  or  information  items. 
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Ratings  of  Areas  of  Wheel  and  Track  Vehicle  Maintenance  Training 

_ TASK  TYPE  AND  RATING  FACTOR 

MAINTENANCE  INSTRUCTIONAL  Motor _ Cognitive  Trblshtng 

LEVEL _ PHASE _ ABCDEF  ABCDEF  ABCDEF 

Power  Plant  341122  331122  211132 

Organizational  Power  Train  441122  421122  221132 

Systems  431132  331132  222343 


Power  Plant  341231  332231  221241 


DS/GS  Power  Train  341231  332231  322341 


Systems  341242  332232  221241 


Table  C-2. 

Ratings  of  Areas  of  Turret  Maintenance  Training 

_ TASK  TYPE  AND  RATING  FACTOR _ 

SKILL  LEVEL  _ Motor _ Cognitive _ Trblshtng 

_ ABCDEF  ABCDEF  ABCDEF 

Organizational  441441  331431  111441 

DS/GS  441441  331431  111441 


Scale  Value 

Key:  1  =  Very  High  A 

2  =  High  B 

3  *  Moderate  C 

4  =  Low  D 

5  =  Very  Low  E 


F 


Factor 

=  Skill  Complexity 
*  Subject-Matter  Difficulty 
=  Commonality  to  Other  Courses 
=  Student  Volume 

=  Effectiveness  of  AET  as  Teaching 
Aid 

=  AET  Cost 


-1^ 
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Area  1  -  Troubleshooting  engines  and  related  systems 

Area  2  -  Troubleshooting  track  vehicle  track/suspension  systems 

Area  3  -  Removal/replacement  of  engines  and  power  packs 

Area  4  -  Troubleshooting  turret  electrical  and  hydraulic  systems 


4  “  'ha  SAG  .a  the  beeis  for  Taeke  3  and 

proceeded  Vccor^kly  ^aaka  3  and  4 
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APPENDIX  D 

EVALUATION  OF  RPF  DEVICES 

The  devices  discussed  in  Section  III  can  be  acceptable  to  the  Army  only 
if  they  are  effective  in  preparing  trainees  to  perform  required  maintenance 
tasks  on  the  job.  To  achieve  this  goal,  the  trainee  must  acquire  equipment 
skills  necessary  to  perform  the  maintenance  tasks,  and  the  cognitive 
knowledge  necessary  to  guide  the  implementation  of  the  skills.  The  training 
device  thus  must  contribute  to  both  skill  acquisition  and  understanding  of 
the  equipment. 

While  training  effectiveness  is  a  requirement  for  a  training  device,  an 
additional  consideration  is  its  cost  effectiveness.  Acceptable  levels  of 
trainee  performance  must  be  achieved  at  a  minimum  training  cost.  The 
evaluation  of  a  device,  therefore,  must  incorporate  both  training 
effectiveness  and  cost  effectiveness  measures. 

The  Army  has  standard  procedures—^  for  identifying  cost  factors  related 
to  training  devices  and  for  incorporating  them  into  training  cost  analyses 
to  determine  user  value.  Hence,  the  focus  of  the  evaluation  methodology 
developed  here  is  on  determining  device  training  effectiveness.  More 
specifically,  the  focus  is  on  determining  the  comparative  effectiveness  of 
alternative  RPF  devices  and  AETs. 

This  Appendix  is  addressed  primarily  to  persona  who  will  be  responsible 
for  device  evaluations.  It  presents  an  outline  of  a  methodology  that  can  be 
adapted  to  the  evaluation  of  any  of  the  devices  discussed  in  Section  III. 
Because  the  methodology  must  be  general  enough  to  include  all  those  devices, 
and  because  they  will  be  evaluated  under  varying  conditions,  details  of  many 
specific  procedures  cannot  be  given.  Even  so,  the  methodology  is 
comprehensive,  and  the  discussion  addresses  technical  issues  that  will 
require  expert  adaptations  to  the  specific  requirements  of  separate 
applications.  The  team  who  will  conduct  the  evaluations  thus  must  include 
the  experts  needed  to  make  the  adaptations. 

In  addition  to  this  introduction,  this  Appendix  includes  three  major 
sections.  The  first  section  presents  an  instructional  context  for  the 
evaluations,  with  special  emphasis  upon  the  structure  of  training  program 
evaluations.  The  second  section  develops  the  evaluation  procedures.  The 
third  section  identifies  the  technical  capabilities  that  must  be  represented 
on  the  evaluation  team. 

In  addition  to  these  three  sections,  three  addenda  appear  at  the  end  of 
this  Appendix.  These  addenda  present  separate  specimen  questionnaires  to  be 
given  to  trainees,  their  AIT  instructors,  and  the  trainees'  job  supervisors 
after  unit  assignment.  The  specimen  questionnaires  concern  gasoline  engine 


— ^Cost  and  Training  Effectiveness  Analysis.  TRADOC  Pamphlet  71-10 
(Draft),  Headquarters,  U.S.  Army  Training  and  Doctrine  Command,  Ft.  Monroe, 
Va.,  1977. 
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maintenance  and  repair,  and  suitable  adaptations  will  be  required  for  other 
types  of  maintenance  training. 

INSTRUCTIONAL  CONTEXT  FOR  THE  EVALUATION 

The  evaluation  of  a  training  device  can  be  viewed  within  the  context  of 
training  course  development.  Various  branches  of  the  military  have  jointly 
developed  a  set  of  procedures,  commonly  referred  to  as  ISD,J:./  which  Includes 
seven  major  steps  to  be  followed  in  conceiving,  developing.  Implementing, 
and  evaluating  a  training  course.  These  steps  will  not  be  detailed  here. 
Instead,  eight  related  or  derivative  procedural  steps  of  particular  concern 
in  training  course  evaluations  will  be  identified  and  discussed  briefly. 
Five  of  these  steps  will  be  discussed  in  detail  in  the  section  that  follows. 

The  eight  procedural  steps  for  evaluations  are  as  follows: 

1.  Statement  of  the  problem.  The  evaluation  problem  as  it  applies  to 
the  particular  case  at  hand  must  state  clearly  the  Issues  to  be  resolved, 
and  in  such  a  manner  that  requirements  for  the  remaining  procedural 
considerations  are  clearly  determined.  The  source  of  the  problem,  and 
Implicit  in  its  statement,  should  provide  a  framework  for  interpreting 
findings  and  deriving  adaptive  actions  based  on  the  findings. 

2.  Analysis  of  cognitive  and  manipulative  skills  to  be  taught. 
Specific  training  objectives  for  the  course  of  concern  must  be  stated  in 
terms  of  those  knowledges  and  skills  required  for  job  performance,  and  for 
which  trainees  are  to  be  prepared  in  the  course. 

3.  Development  of  the  training  course.  This  step  encompasses  the 
design  of  the  instructional  process  whereby  course  objectives  are  to  be 
achieved.  Of  particular  concern  for  the  present  effort  is  the  assurance 
that  the  instructional  program  is  designed  to  capitalize  on  the  specific 
training  capabilities  of  the  various  devices  being  used. 

4.  Selection  of  criterion  measures.  Psychometrically  sound  measuring 
instruments  and  procedures  must  be  available  for  determining  as  objectively 
as  possible  the  extent  to  which  course  objectives  are  achieved  by  trainees. 
In  the  context  of  device  evaluation,  it  is  especially  important  that 
measures  of  device-relevant  objectives  be  those  which  the  device,  by  design 
and  utilization,  can  help  achieve.  In  addition  to  measuring  trainee 
achievements,  the  evaluation  of  a  device  requires  information  regarding 
device  use  and  its  acceptance  by  users. 

5.  Design  of  the  evaluation.  All  independent  variables  of  concern  in 
the  evaluation  must  be  identified  and  their  roles  must  be  implemented  in 
such  a  way  that  the  data  to  be  collected  reflect  cleanly  the  effects  ot  each 
such  variable.  In  addition,  provisions  must  be  made  for  controlling  the 
ef fee  ts 


JL/ Interservice  Procedures  for  Instructional  Systems  Development. 
TRADOC  Pamphlet  350—30,  Headquarters,  U.S.  Army  Training  and  Doctrine 
Command,  Ft.  Monroe,  Va.,  1  August,  1975. 
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effects  of  variables  that  are  not  in  themselves  of  concern,  but  which,  if 
not  controlled,  can  prevent  a  clear  identification  of  effects  of  the 
independent  variables. 

6.  Collection  of  data.  This  step  must  provide  for  the  administration 
of  all  measuring  instruments.  The  sources  of  data  must  be  specified  and  a 
schedule  for  the  collection  established. 

7.  Analysis  of  data.  Certain  specific  aspects  of  data  analysis  must 
be  determined  by  the  types  of  data  available,  and  by  the  results  of  other 
data  analyses.  Nevertheless,  the  structure  for  examining  data  should  be 
clear  at  the  outset,  and  a  basic  analytic  schema  established. 

8 .  Integration  of  findings  vis-a-vis  the  evaluation  problem.  As 
stated  in  connection  with  1  above,  a  framework  for  interpreting  findings, 
and  for  deriving  adaptive  actions  based  on  them,  should  be  the  source  of  the 
problem  statement  originally.  Hence,  the  procedures  involved  in  this  step 
are  to  be  directed  toward  resolving  issues  which  gave  rise  to  the  evaluation 
effort,  using  the  information  obtained  from  that  effort. 

EVALUATION  PROCEDURES 

Three  of  the  procedural  steps  above  go  beyond  the  scope  of  the  present 
effort.  Specifically,  Steps  2,  3,  and  8  will  be  part  of  the  final 
evaluation  effort,  and  the  previously  referenced  ISD  procedures  provide 
adequate  guides  for  their  completion.  Hence,  they  will  not  be  discussed 
further  in  this  report  except  for  general  considerations  concerning  them  as 
they  apply  to  the  remaining  steps. 

Steps  1  and  4-7,  or  certain  aspects  of  them,  are  of  particular  concern 
in  this  project,  however.  They,  too,  are  included  in  ISD  procedures,  but 
Task  5  of  this  project  calls  for  adapting  these  steps  to  the  unique  problems 
of  evaluating  the  maintenance  training  devices  identified  in  this  report. 
The  remaining  parts  of  this  section  will  discuss  these  five  steps  in  order, 
focusing  upon  their  adaptations  to  the  evaluation  of  the  maintenance 
training  devices. 

Statement  of  the  Problem 


This  step  has  two  facets,  as  pointed  out  above.  The  first  is  to 
provide  a  guide  for  deriving  the  remaining  procedural  steps;  and,  second,  by 
its  derivation,  it  should  provide  a  framework  for  interpreting  evaluative 
data  and  for  determining  adaptive  actions  based  on  the  data.  The  second 
facet  is  covered  in  the  main  text  of  this  report,  specifically  in  Section  I 
in  discussions  of  the  need  for  better,  cost-effective  maintenance  training, 
the  military  problem  of  providing  such  training,  and  the  research  problem 
prompted  by  the  need  and  the  military's  reaction  to  it. 

The  first  facet  of  the  problem  statement  is  a  clear  definition  of  the 
issues  to  be  resolved,  stated  so  as  to  establish  requirements  for  the 
remaining  procedural  steps.  For  present  purposes,  the  problem  is  to 
determine  the  training  effectivene^  of  certain  alternative  maintenance 
training  devices  relative  to  corresponding  AETs  and,  in  some  cases,  to 
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alternative  devices.  The  problem  requires  that  research  procedures  be 
established  for  answering,  or  otherwise  resolving,  seven  questions  regarding 
training. 

These  questions  are; 

1.  What  kinds  of  measures  can  be  used  that  are  indicative  of 
training  effectiveness? 

2.  From  what  sources,  under  what  conditions,  and  from  whom  should 
these  measures  be  obtained? 

3.  How  are  the  measures  to  be  analyzed  so  a^  to  provide  comparative 
indices  of  RPF  and  AET  training  effectiveness? 

4.  Is  the  effectiveness  of  a  device,  including  AET,  dependent  upon 
certain  instructional  modes? 

5.  Is  the  effectiveness  of  a  device  per  se,  or  its  use  in  alternative 
instructional  modes,  dependent  upon  certain  student  characteristics 
such  as  aptitude  level  or  amount  of  relevant  experience? 

6.  Because  instructors  may  vary  in  the  effectiveness  with  which  they 
use  various  devices  in  training,  how  are  instructor  effects  to  be 
provided  for  in  the  analyses? 

7.  Because  training  sites  may  differ  in  ways  affectinj  device  use  and 
training  value,  how  are  the  effects  of  these  differences  to  be 
determined  or  otherwise  provided  for  in  the  analyses? 

As  it  applies  to  the  present  problem,  question  1  is  answered  in  the 
subsection  that  follows.  The  various  aspects  of  question  2  are  treated  in 
the  next  subsection,  as  is  question  6.  The  remaining  questions  then  are 
addressed  in  the  following  subsection. 

Selection  of  Criterion  Measures 


The  primary  concern  in  evaluating  training  devices  is  the  adequacy  of 
on-the-job  performance  of  graduates  of  training  programs  that  use  the 
devices.  It  is  necessary  therefore  that  job  competence  of  graduates  be 
assessed,  and  in  a  manner  that  reveals  the  adequacies,  and  lacks  thereof,  of 
devices  used  in  preparing  them  for  unit  responsibilities.  Time  constraints 
require  that  such  assessments  be  made  soon  after  prototype  iLaining  devices 
become  available,  for  decisions  regarding  the  procurement  of  additional 
devices  must  be  made  without  undue  delay.  Hence,  job  performance  measures 
that  are  revealing  of  competence,  and  that  can  be  obtained  relatively  soon 
after  training,  must  be  used. 

A  second  set  of  measures,  a  group  administered  during  and  immediately 
upon  completion  of  AIT,  should  also  be  obtained.  Such  measures  can  reveal 
specific  strengths  and  weaknesses  of  training  programs  according  to 
attainments  of  particular  training  objectives.  The^e  indic.itors  would  not 
only  reveal  trainee  achivement  during  training  as  it  relates  to  device  use. 
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but  to  the  extent  that  such  achievement  is  predictive  of  later  job 
performance,  they  provide  data  within  the  time  constraint  that  can  be  used 
in  decisions  regarding  device  procurement. 

Thus,  these  two  sets  of  measures,  one  of  job  performance  and  the  other 
of  AIT  achievement,  will  permit  evaluations  of  devices  as  they  affect  both 
the  achievement  of  particular  AIT  objectives  per  se,  and  the  transfer  of 
knowledge  and  skills  related  ir  hose  objectives  to  th^  complex  integrations 
of  behaviors  required  for  adeq,_.  ce  job  performance.  — ' 

The  kinds  of  measures  required  for  these  dual  evaluations  are  discussed 
in  detail  below.  Also,  separate  discussions  are  devoted  to  two  other  types 
of  measures  that  will  be  needed.  One  concerns  user  acceptance  of  the 
devices  and  the  training  programs  in  which  they  are  used.  The  second  type 
of  additional  measure  concerns  information  regarding  device  use  and 
characteristics  which  can  aid  in  interpreting  device  effects  on  training, 
and  can  provide  inputs  for  cost  analyses. 

AIT  achievement  measures.  To  be  useful  for  evaluating  training 
devices,  measures  obtained  during  or  at  the  end  of  AIT  (as  well  as  measures 
of  job  performance  must  satisfy  two  criteria:  (1)  they  should  focus  upon 
aspects  of  achievement  which  are  relevant  to  device  design  and  utilization; 
and  (2)  they  should  be  psychometrically  sound.  The  application  of  these 
criteria  will  be  discussed  briefly  before  turning  to  the  nature  of  the  AIT 
measures  themselves. 

The  devices  to  be  evaluated  will  be  used  for  only  certain  segments 
(e.g.,  gasoline  engine  maintenance)  of  overall  AIT  training.  Furthermore, 
among  the  training  objectives  included  in  these  particular  segments,  the 
achievement  of  only  certain  ones  can  reasonably  be  expected  to  be  enhanced 
by  given  devices.  Hence,  at  the  outset,  device-relevant  objectives  should 
be  identified,  and  provisions  should  be  made  for  separating  measures  of 
their  attainment  from  measures  concerned  with  objectives  not  related  to  the 
purposes  implied  by  device  design  and  use.  Device-relevant  measures  can  be 
administered  along  with  the  others,  however;  the  reqjirement  is  only  that 
they  be  quantitatively  distinguishable. 

The  second  criterion,  psychometrics  1  soundness,  applies  not  only  to  AIT 
achievement  and  job  performance  measures,  but  to  all  measures  obtained  for 
the  evaluations.  It  should  be  ascertained  that  each  separate  measurement 
item  or  procedure  of  concern  is  conceived  and  formulated  in  a  way  to  assure 
validity  and  objectivity  in  measuring  the  particular  objective  for  the  item 
or  procedure.  Generally,  validity  can  be  attained  by  appropriate  logical 
extensions  of  separate  training  objectives  into  statements  of  observable 
behavior  or  other  relevant  manifestations.  Objectivity  is  usually  attained 
to  the  extent  that  the  manifestations  can  be  observed  and  quantified 
unambiguous ly . 


—  Certain  devices  may  on  occasion  be  used  for  unit  training  as  well  as 
for  AIT.  In  such  cases,  points  made  relative  to  AIT  would  need  adapting  to 
the  requirements  and  conditions  of  unit  courses. 
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Exceptions  to  rigorous  objectivity  should  be  restricted  primarily  to 
measures  obtained  through  questionnaires.  As  discussed  elsewhere, 
instruments  of  this  type  are  to  be  used  to  measure  user  acceptance  of 
training  programs,  some  aspects  of  training  achievement  and  on-the-job 
performance,  and  certain  other  relevant  variables.  The  information  to  be 
obtained  is  Inherently  subjective.  It  is  imperative  therefore  that  schemata 
for  recording  data  provide  for  maximum  reliability  of  subjective  reports. 
Both  the  instruments  for  recording  these  judgments,  and  the  instructions 
accompanying  them,  should  help  the  responder  focus  on  clear-cut 
characteristics  to  be  rated,  and  provide  meaningful  criteria  for  assigning 
ratings. 

Two  types  of  achievement  should  be  measured  during  or  at  the  end  of 
AIT,  knowledge  or  cognitive  skills,  and  manipulative  or  hands-on  skills. 
The  former  skills  Include  theory  of  equipment  construction  and  functioning, 
as  well  as  those  aspects  of  procedural  knowledge  required  to  guide  the 
application  of  hands-on  skills.  The  hands-on  skills  include  all  relevant 
manipulative  actions  that  must  be  taken  to  complete  given  maintenance  tasks. 

Both  types  of  achievement  are  currently  mer.sured  in  existing  training, 
programs,  and  it  is  likely  that  some  measures  as  now  obtained  could  be 
readily  adapted  to  evaluations  of  training  devices.  However,  the  criterion 
of  device  relevance  should  be  applied  in  their  selection  and  modification  as 
discussed  above.  It  is  also  likely  that  some  device-relevant  objectives  are 
not  measured  by  existing  tests.  This  lack  could  be  due  to  objectives 
important  for  device  evaluations  simply  being  omitted  from  samples  of 
achievement  areas  covered  by  existing  tests.  Or,  the  lack  may  be  because 
certain  devices  make  practicable  the  achievement  of  objectives  not  included 
in  existing  courses.  In  either  case,  entirely  new  test  items  and  procedures 
would  have  to  be  developed^  for  the  instruments  and  procedures  finally  used 
for  evaluation  should  tap  all  important  device-relevant  objectives. 

Once  relevant  objectives  are  identified,  both  existing  test  items,  and 
new  ones  as  needed,  should  be  examined  for  psychometrical  soundness. 
Generally,  each  objective  should  be  logically  extended  to  observable, 
quantifiable  manifestations.  For  example,  an  objective  might  state  that  a 
trainee  will  be  able  to  identify  symptoms  of  a  fuel  system  malfunction.  If 
this  objective  is  achieved,  the  trainee  should  also  be  able  to  state  the 
nature  of  these  symptoms  when  properly  questioned,  or  to  recognize  and 
discriminate  their  occurrence  in  a  malfunctioning  AET  or  as  simulated  in  an 
RPF.  In  either  case,  measures  could  be  derived  as  the  number  of  symptoms 
correctly  identified,  the  number  of  inapplicable  symptoms  falsely 
Identified,  and/or  the  numbers  of  Instances  in  which  target  symptoms  are 
confused  with  irrelevant  ones.  Thus,  the  trainees'  achievement  could  be 
quantified  as  numbers  of  errors  or  of  appropriate  identifications.  The 
length  of  time  required  to  identify  symptoms  would  also  reflect  some  aspects 
of  achievement.  Furthermore,  for  training  objectives  permitting  clear-cut 
categorical  assessment  (e.g.,  go/no-go  decisions  for  a  series  of  objectives 
to  be  attain 'id  in  sequence),  rate-based  measures  can  provide  the  necessary 
quantification.  Possibilities  include  time  required  per  objective 
attainment,  number  or  attainments  per  unit  of  time,  or  simply  the  total 
number  of  objectives  achieved  during  the  portion  of  the  course  devoted  to 
relevant  topics. 
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Job  performance  measures.  Adequacy  of  job  performance  is  only 
partially  determined  by  cognitive  and  hands-on  skills  per  se.  Such  skills 
are  necessary,  but  not  sufficient  by  themselves,  because  they  must  be 
integrated  into  appropriate  judgeirents  and  actions  as  determined  by  the  cues 
and  demands  of  real-world  situations.  In  addition,  the  graduate  trainee  is 
expected  to  progress  on  the  job  beyond  the  achievement  levels  attained  in 
AIT.  Hence,  indicators  of  job  performance  must  tap  capabilities  and  actions 
that  cannot  be  observed  during  AIT. 

This  is  not  to  say  that  AIT  types  of  achievement  should  not  be  measured 
on  the  job.  Information  regarding  the  retention  of  skills  learned  in 
earlier  training  is  needed  to  help  interpret  the  more  complex  aspects  of  job 
performance.  For  example,  the  extent  to  which  previous  learning  can  be 
integrated  appropriately  for  real-world  performance  will  depend  on  its 
retention  in  some  form,  so  the  relative  permanence  of  AIT  learning  should  be 
determined  for  each  training  program.  Furthermore,  subsequent  learning  on 
the  job  will  depend  to  some,  probably  a  great,  extent  upon  retention  of 
previous  knowledge  on  which  to  build. 

Thus,  evaluative  measures  ob^aine(^  on  the  job  should  focus  both  on 
cognitive  and  hands-on  achievement  as  in  AIT,  and  on  the  more  complex 
aspects  of  operational  competence.  (It  is  recognized  that  scores  on  the 
latter  measures  will  be  determined  by  factors  such  as  trainee  motivation  and 
attitudes  toward  their  job  responsibilities,  and  that  such  influences  can 
represent  "noise"  in  measures.  Nevertheless,  these  influences  can  be 
minimized,  or  otherwise  controlled,  by  judicious  selections  of  behaviors  to 
be  measured.  To  the  extent  that  job  motivation  and  attitudes  reflect 
effectiveness  of  training,  these  influences  would,  of  course,  not  be 
considered  noise.) 

As  stated  above,  cognitive  and  hands-on  achievement  should  be  assessed 
on  the  job.  The  Skill  Qualification  Test  (SQT),  normally  administered  six 
months  after  completing  AIT,  is  one  readily  available  source  for  such 
measures.  However,  other  achievement  measures  should  be  obtained 
approximately  two  months  following  AIT.  These  latter  measures  will  probably 
reflect  AIT  achievement  less  ambiguously,  in  that  AIT  and  job  learning  will 
be  less  confounded.  For  comparability  of  data,  it  would  be  desirable  at 
thito  two-month  point  to  include  a  pencil-and-paper  test  similar  to  that  used 
in  AIT  for  retention  of  cognitive  learning,  as  well  as  measures  of  hands-on 
skills  which  overlap  with  those  used  in  AIT. 

Because  of  its  complex  nature,  job  performance  beyond  individual 
manipulative  skills  must  be  measured  according  to  criteria  that  reflect  that 
complexity.  Expansion  of  job  capabilities  as  mentioned  earlier  can  be 
assessed  via  questionnaires  completed  by  supervisors.  For  example,  current 
skills  or  skill  levels  not  evident  at  the  beginning  of  unit  assignment  coaid 
be  identified  by  a  listing  of  the  numerous  relevant  skills  needed  for  job 
performance,  with  a  rating  scale  for  each  on  which  growth  in  competence 
could  be  indicated.  Other  questions  could  be  designed  to  identify  specific 
trainee  strengths  and  weaknesses,  the  supervisors'  overall  evaluations  of  or 
confidence  in  the  trainees'  capabilities,  etc.  In  addition  to  supervisors' 
evaluations,  analogous  self-evaluations  by  trainees  should  be  obtained. 
(Items  illustrating  these  types  of  measures  appear  in  the  specimen 
questionnaires  in  Addenda  1  and  3  to  this  Appendix.) 
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In  addition,  a  number  of  objective  measures  could  be  used  to  assess  job 
performance  if  they  are  determined  under  standard  conditions.  Number  of 
attempts  or  time  required  to  successfully  complete  a  maintenance  task; 
dowiitime  of  vehicles  needing  repair;  immediate  recurrence  of  similar 
symptoms  of  malfunctions  following  equipment  repair,  or  occurrence  of  other 
symptoms  that  indicate  inadequate  repair;  checks  and  confirmations  by 
supervisors  that  a  system  has  been  adjusted  or  repaired  to  a  specified 
maintenance  level;  numbers  of  system  components  returned  from  the  field  for 
repair  that  do  not  actually  need  it — these  and  other  objective  indicators 
can  provide  evidence  of  adequacy  of  performance.  However,  such  measures  can 
be  ustiful  only  if  individual  responsibility  for  the  related  actions  can  be 
established.  Hence,  procedures  for  gathering  such  data  must  permit 
identification  of  individuals  involved  and  their  separate  roles. 

Specifications  for  individual  measures  must  await  the  task  analyses  and 
the  establishment  of  training  requirements  referred  to  earlier.  Whatever 
form  they  finally  take,  however,  they  should  conform  to  the  criteria  of 
device  relevance  and  psychometric  soundness. 

User  acceptance  measures.  The  focus  for  user  acceptance  measures 
should  be  upon  the  satisfaction  of  trainees  and  instructors  with  training 
devices  used,  and  upon  job  supervisors'  satisfaction  with  trainees  after 
training.  Because  the  value  of  devices  depends  to  a  great  extent  upon  how 
they  are  used,  acceptance  measures  should  also  reflect  satisfaction  with  the 
relevant  segments  of  the  training  program  as  well. 

Device  acceptance  by  trainees,  instructors,  and,  by  implication, 
supervisors  is  of  concern  for  two  reasons.  First,  it  is  important,  although 
perhaps  not  crucial,  for  morale  that  users  of  given  programs  and  media  see 
them  as  valuable  and  their  roles  in  them  as  satisfying.  Second,  in  the 
event  that  a  training  device  is  or  could  be  training  effective,  but 
generally  is  not  seen  as  satisfactory  by  groups  of  users,  establishing  its 
nonacceptability  would  provide  a  signal  for  possible  device  design  changes 
or  for  needed  managerial  actions  directed  toward  educating  the  users 
regarding  the  device's  value. 

Most,  if  not  all,  data  concerning  user  acceptance  should  be  gathered 
through  carefully  constructed  questionnaires.  Each  of  the  three  specimen 
questionnaires  that  appear  as  addenda  to  this  Appendix  provides  for  such 
measures.  The  questions  for  instructors  are  the  most  detailed,  because  they 
will  know  about  aspects  of  device  desigii  and  use  that  trainees  and 
supervisors  will  not  know.  The  questions  for  supervisors  are  necessarily 
general,  related  to  overall  quality  of  training,  because  they  may  not  even 
know  that  a  device  was  used  in  AIT. 

As  illustrated  in  the  specimen  questionnaires,  user  acceptance  measures 
would  be  part  of  the  same  instruments  used  to  gather  data  for  some  job 
performance  evaluations,  and  for  other  facets  of  the  evaluation  discussed  in 
the  next  paragraph. 

Additional  measures  needed.  Other  types  of  data,  specifically 
regarding  device  use,  will  be  needed  for  an  adequate  evaluation  of  each 
device.  These  data  can  serve  two  purposes.  Some  cm  provide  information 
that  will  aid  in  interpreting  various  results  of  training  effectiveness 


80 


analyses.  A  second  group  of  measures  can  provide  input  for  certain  training 
cost  analyses. 

Data  that  can  aid  in  interpreting  effects  of  training  include  trainee 
attrition  rate;  amount  and  nature  of  device  use;  instructors'  and  trainees' 
evaluations  of  devices;  level,  depth,  and  comprehensiveness  of  training 
attempted  with  the  aid  of  devices;  adequacy  of  software  used  with  devices; 
amount  and  nature  of  instructor  effort  required;  etc.  Such  data  may  also 
reflect  user  acceptance.  Data  such  as  attrition  rate  can  be  obtained  from 
school  records.  Much  of  the  other  information  can  be  gathered  with 
questionnaires  as  illustrated  in  Addenda  1  and  2  to  this  Appendix. 

Data  needed  for  cost  analyses  include  mean  time  required  to  complete 
each  program,  which  is  of  special  interest  in  comparing  lock-step  and 
individually  paced  programs;  numbers  of  instructors  and  amounts  of 
instructor  effort  required  in  the  various  programs;  equipment  set-up  time; 
etc.  Some  measures  of  these  sorts  are  included  in  the  questionnaire  for 
instructors  (Addendum  2);  others  would  be  obtainable  from  official  records. 
Generally,  however,  the  cost  analysis  procedures  previously  refe^*»nced 
should  be  the  primary  guide  for  the  kinds  of  cost-related  measures  needed. 

Design  of  the  Evaluation 

The  training  device  is  the  independent  variable  of  primary  concern. 
Hence,  the  focus  for  the  evaluation  is  on  the  comparison  of  the  training 
effectiveness  of  various  RPF  devices  with  corresponding  AETs,  and  when  they 
are  available,  the  comparison  of  alternative  RPF  devices.  Important  to 
these  comparisons  are  the  effects  of  using  the  devices  (including  AETs)  in 
lock-step  as  opposed  to  self-paced  programs.  In  addition,  the  effectiveness 
of  each  device  for  trainees  of  different  aptitude  levels  or  amounts  of 
relevant  experience  is  of  concern.  Hence,  data  should  be  collected  that 
permit  meaningful  comparisons  of  device  effectiveness  separately  by 
instructional  mode  and  by  these  trainee  characteristics. 

Assignment  of  trainees.  These  comparisons  require  that  separate 
training  groups  be  established  for  each  device-instructional  mode 
combination.  Within  each  group,  all  levels  of  trainee  characteristics  are 
to  be  represented.  (See  Figure  D-1  below  for  graphic  representation  of  the 
design.)  A  parent  group  of  trainees  at  each  training  site  should  be 
stratified  according  to  aptitude  and/or  experience  levels,  and  then  assigned 
randomly  from  within  each  stratum  to  the  various  training  programs. 
Matching  of  trainees  within  strata  prior  to  assignment  should  be  avoided, 
however,  because  matching  would  introduce  correlations  among  training  groups 
that  would  unduly  complicate  the  analyses. 

Assignment  of  instructors.  Potentially,  the  greatest  difficulty  in 
interpreting  data  from  an  evaluation  study  as  proposed  here  is  the 
confounding  of  instructor  and  program/device  effects.  Instructors  are  not 
likely  to  be  equally  effective  for  a  give’-  type  of  program,  nor  will  a  given 
instructional  mode  device  that  is  supeiior  when  used  by  one  instructor  be 
necessarily  superior  when  used  by  another.  Hence,  instructors  must  be 
assigned  in  such  a  way  that  their  unique  effects  can  be  safely  ignored  in 
the  analyses. 
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Basic  ANOVA  design  for  analyzing  device  evaluation  data 
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One  such  assignment  procedure  would  require  a  pool  of  instructors  from 
which  individuals  could  be  drawn  for  random  assignment  to  the  various 
training  programs.  Each  instructor  would  participate  in  only  one  program, 
but  each  program  would  have  multiple  instructors  for  multiple  groups  of 
trainees.  Combined  instructor-program  effects  on  trainee  achievement  would 
then  be  randomized  across  training  classes.  Unless  training  site 
differences  can  be  safely  ignored  or  controlled  statistically,  this 
assignment  procedure  would  not  be  practicable  except  at  individual  sites 
that  had  several  instructors  available  for  multiple  classes  in  two  or  more 
types  of  programs. 

A  second  procedure  for  controlling  for  instructor  effects  would  entail 
the  use  of  each  instructor  in  each  local  program.  Instructors  would  then 
comprise  an  additional  dimension  in  the  basic  ANOVA  design  (see  above),  and 
their  unique  influences  could  be  determined  directly  and  used  as  needed  in 
interpreting  achievement  data. 

It  is  possible  that  neither  procedure  for  assigning  instructors  can  be 
followed  completely,  at  least  at  all  training  locales.  If  such  be  the  case, 
interpretations  of  achievement  data  may  not  be  clear  cut,  regardless  of 
significance  levels  of  statistical  tests.  Also,  the  assignment  of  some,  but 
not  all,  instructors  to  different  programs  would  result  in  some  between- 
group  correlations  of  achievement  data  that  would  have  to  be  recognized 
during  data  analyses. 

Standardization  of  training.  Ideally,  the  training  program  used  with 
each  device  should  be  tailored  to  capitalize  maximally  on  the  training 
potential  of  the  device.  Such  optimum  training  conditions  will  not  have 
been  established  empirically  at  the  time  of  the  original  device  evaluations. 
Hence,  those  responsible  for  the  evaluations  must  ensure  that  device  use 
conforms  to  applicable  principles  of  learning  technology.  With  the 
exception  immediately  following,  a  standard  training  regimen  should  thus  be 
established  for  each  type  of  device  which  incorporates  device  training 
characteristics  in  an  effective  manner. 

The  exception  just  referred  to  concerns  differences  in  the 
instructional  modes,  which  are  of  considerable  importance  in  the  evaluation. 
For  any  device,  dual  regimens  will  be  required,  one  for  lock-step  and  one 
for  individually  paced  instruction.  However,  except  for  differences 
entailed  by  the  modes  per  se,  the  dual  regimens  should  be  identical. 

In  order  to  provide  meaningful  bases  for  comparisons,  once  a  regimen  is 
established  for  a  given  device  and  mode,  the  investigators  should  ensure 
that  all  instructors  using  the  device  faithfully  follow  the  prescribed 
trai  ;g  regimen. 

Collection  of  data.  Achievement  data  should  be  collected  on  each 
trainee  at  the  end  of  AIT,  and  during  AIT  as  relevant  data  are  available. 
After  approximately  two  months  of  unit  assignment  following  AIT,  job 
performance  data  for  each  trainee  should  be  obtained  from  job  supervisors 
and  otherwise  as  discussed  above.  In  addition,  trainee  performance  on  the 
Skill  Qualification  Test,  normally  taken  six  months  after  unit  assignment, 
will  provide  additional  data  regarding  knowledge  and  skill  levels,  and  some 
aspects  of  job  performance. 
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I'ser  acceptance  data  should  be  obtained  from  trainees  immediately 
following  AIT,  and  again  from  them  and  from  job  supervisors  approximately 
two  months  following  unit  assignment.  Similar  data  should  be  obtained  from 
instructors  at  the  end  of  the  AIT  course  used  in  the  evaluations.  Also  at 
that  time,  the  information  discussed  above  regarding  characteristics  and 
utilization  of  training  devices  could  be  obtained  from  the  instructors. 

Table  D-1  presents  a  schedule  for  obtaining  these  measures.  The 
measures  are  identified  only  as  general  classes.  The  subsection  above 
describing  measures  to  be  obtained  identifies  subc lasses  of  each  general 
type. 


Table  D~1 

Schedule  for  Obtaining  Measures 


CLASS  OF  MEASURE 


Six  Months 

During  End  of  Two  Months  post-AIT 
AIT _ AIT _ post-AIT _ (SQT) 


AIT  Achievement/Retention: 

Equipment  knowledge  X 

Procedural  knowledge  X 

Hands-on  skills  X 

User  Acceptance  by: 

Trainee 

Instructor 

Supervisor 

Device  Characteristics/Use 


X 

X 

X 


X 

X 

X 


X 

X 

X 


X 

X 

X 


X 

X 

X 


Analysis  of  Data 

Questions  3,  4  and  5  on  page  D-4  concerned,  respectively,  the 
comparative  effectiveness  of  RPF  devices  and  AETs ,  their  effectiveness  when 
used  with  different  instructional  modes,  and  their  effectiveness  as 
dependent  upon  certain  student  characteristics.  The  basic  analytic  design 
presented  below  provides  for  direct  determination  of  answers  to  those 
questions. 

The  basic  design  is  a  three-way  analysis  of  variance  (ANOVA)  as  shown 
in  Figure  D-1  above.  One  dimension  of  this  schema  provides  for  comparisons 
of  the  media  of  primary  concern,  i.e.,  AET  and  one  or  mere  RPF  devices. 
Although  only  a  single  RPF  device  is  represented  in  the  figure,  if 
comparable  data  are  available  for  two  RPF  devices,  a  second  RPF  row  can  be 
added  to  the  Device  classification.  On  the  other  hand,  when  data  are 
available  for  only  one  type  of  media,  this  dimension  will  not  appear  in  the 
analysis . 


k 


84 


Columns  in  the  ANOVA  model  identify  types  of  instructional  mode  such  as 
lock-step  or  self-paced  instruction.  Since  the  adaptability  of  a  device  to 
the  self-paced  instructional  mode  is  of  considerable  concern  in  the  present 
context,  this  dimension  of  the  analysis  is  of  considerable  importance. 
However,  here  too,  it  may  be  that  data  for  more  than  two,  or  only  one,  mode 
are  available  in  particular  instances.  In  such  cases,  the  dimension  can  be 
extended  or  collapsed  as  described  for  the  Device  dimension. 

The  third  dimension  of  the  basic  design  represents  levels  of  trainee 
characteristics.  When  the  necessary  data  are  available,  trainees  should  be 
classified  according  to  level  of  aptitude  scores,  or  according  to  amount  of 
previous  relevant  experience.  (Stratification  of  trainees  according  to  both 
aptitude  and  experience  levels  would  require  an  additional  Trainee 
Characteristic  dimension  in  the  basic  design.)  Only  two  levels  for  a  single 
type  of  characteristic  are  shown,  but  it  would  be  desirable  to  have  four  if 
trainee  variability  is  sufficiently  great. 

The  primary  concern  in  this  evaluation  is  the  relative  effectiveness  of 
alternative  devices  used  in  training,  whether  different  RPFs  or  AETs .  The 
ANOVA  provides  for  such  comparisons  overall,  i.e.,  regardless  of 
instructional  mode  or  trainee  characteristic,  as  well  as  comparisons  of 
device  effectiveness  separately  by  instructional  mode  and/or  trainee 
characteristic. 

The  complexity  of  the  basic  design  should  be  increased  in  some 
instances,  or  other  classifications  or  dimensions  substituted  for  those 
shown  in  Figure  D-1,  to  provide  additional  information  needed  to  understand 
device  effectiveness.  Two  likely  dimensions  are  discussed  below. 

Repeated  measures  of  trainee  achievement.  One  such  dimension  would 
provide  for  repeated  measures  of  a  similar  kind  on  the  same  trainees. 
Repeated  measures  are  needed  to  answer  questions  regarding  patterns  of 
change  in  trainee  achievement  over  time,  and  as  related  to  devices  used  in 
training.  For  example,  retention  of  knowledge  and  skills  after  unit 
assignment  may  well  vary  with  type  of  device  used  in  training,  together  with 
level  of  trainee  characteristic  and/or  instructional  mode.  Hence,  identical 
measures  taken  at  the  end  of  AIT  and  after  2  months  of  unit  experience  would 
comprise  two  classifications  in  a  repeated  measures  dimension.  Patterns  of 
training  group  means  of  these  measures  could  reveal  differential  retention 
according  to  type  of  training  device  used  and/or  its  joint  effects  with 
instructional  mode  and  student  characteristic. 

Progress  during  AIT  may  also  vary  with  devices  used,  instructional 
mode,  and  trainee  characteristics.  Successive  measures  of  given  types  of 
achievement  could  thus  be  made  during  AIT  and  analyzed  as  repeated  measures. 
Information  regarding  device  effects  would  be  analogous  to  that  in  the 
preceding  paragraph. 

Training  site.  Another  dimension  that  may  need  to  be  added  to  the 
basic  design,  or  replace  one  in  the  design,  is  the  training  locale. 
Training  programs  at  different  sites,  or  under  AIT  as  opposed  to  unit 
administration,  can  thus  be  compared.  However,  because  in  such  instances 
random  assignments  of  trainees  cannot  be  made  from  a  single  parent  group  as 
discussed  above,  it  will  be  especially  important  to  stratify  trainees 
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according  to  both  aptitude  and  experience  levels,  thus  providing  two 
dimensions  of  trainee  characteristics.  The  double  stratification  would 
enstire  comparability  of  trainees  on  at  least  two  sets  of  characteristics. 
Results  of  such  analyses  will  not  be  as  definitive  as  random  assignments 
from  a  single  parent  group  would  permit,  but  cautiously  interpreted  they 
could  still  provide  insights  into  device  effects  not  otherwise  available. 

This  basic  design,  with  extensions  as  described,  is  to  be  used  for  all 
dependent  data  in  a  suitable  metric.  The  design  can  be  adapted  to  chi 
square  or  other  nonparametric  analyses  for  dependent  measures  of  categorical 
nature.  Generally,  only  two-  and  three-dimensional  chi  square  tables  would 
be  used,  with  one  or  two  of  the  dimensions  drawn  from  the  ANOVA  model  and 
the  second  or  third  from  score  categories. 

SUPPORT  PERSONNEL 

Seven  types  of  technical  capabilities  must  be  represented  on  the  team 
that  conducts  the  training  evaluations.  These  capabilities  are: 

1.  Expertise  in  administering  training  program  evaluations ; 

2.  Expertise  in  job/task  analyses  and  in  deriving  training 
requirements  from  them; 

3.  Expertise  in  training  course  design,  with  special  emphasis  on 
training  device  utilization; 

4.  Expertise  in  developing  measuring  instruments; 

5.  Expertise  in  experimental  design,  especially  the  adaptation  of 
design  requirements  to  varying  local  training  conditions; 

6.  Expertise  in  data  analysis  in  general,  and  in  ANOVA  and  chi  square 
analyses  in  particular;  and 

7.  Technical  expertise  in  subject  matter. 
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ADDENDA  TO  APPENDIX  D 


^  Three  addenda^  to  Appendix  D  follow.  Each  is  a  specimen  questionnaire 
designed  to  elicit  information  regarding  training  on  gasoline  engine 
maintenance  and  repair.  The  first  Addendum  is  a  questionnaire  foe  trainees, 
the  second  is  for  their  instructors,  and  the  third  is  for  job  supervisors  of 
graduate  trainees.  Preceding  the  questionnaires  is  a  specimen  form  for 
providing  data  required  by  the  Privacy  Act  of  1974.  Properly  completed,  a 
copy  of  this  form  would  accompany  each  questionnaire. 
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DATA  REOUIRFD  BY  THE  PRIVACY  ACT  OF  1974 

li  If  S.C  Ml'al 

TO  l.r  Of  f  OHM 

PHl&CHIMINC  DIHEC' 

AR  70-1 

1.  authority 

a.  raiNciPAL  purpose(S) 

The  data  collected  with  the  attached  form  are  to  be  used  for  research 
purposes  only. 


3.  SOUTINE  USES 


This  Is  an  experimental  personnel  data  collection  form  developed  by 
pursuant  to  Its  research  mission  as  prescribed  In 

When  identifiers  (name  or  Social  Security  Number)  are  requested  they  are  to 
be  used  for  administrative  and  statistical  control  purposes  only.  Full 
confidentiality  of  the  responses  will  be  maintained  in  the  processing  of 
these  data. 


Signature 


4.  MAh/DATORY  OR  VOLUNTARY  DISCLOSURE  AND  EFFECT  ON  INDIVIDUAL  NOT  PROVIDING  INFORMATION 

Your  participation  in  this  research  is  strictly  voluntary.  Individuals 
are  encouraged  to  provide  complete  and  accurate  information  in  the  interests  of 
the  research,  but  there  will  be  no  effect  on  individuals  for  not  providing  all 
or  any  part  of  the  information.  This  notice  may  be  detached  from  the  rest  of 
the  form  and  retained  by  the  individual  if  so  desired. 
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FORM 


1.  Sp<!cimen  Trainee  Questionnaire  for  Evaluating  Training  for 
The  Maintenance  and  Repair  of  Gasoline  Engines 


The  purpose  of  this  questionnaire  is  to  determine  how  you  feel 
regarding  your  training  and  ability  to  maintain  and  repair  gasoline  en¬ 
gines.  Four  types  of  information  are  covered.  First,  there  are  questions 
asking  how  well  prepared  you  feel  you  are  to  do  the  maintenance  and  repairs 
on  gasoline  engines  that  are  expected  of  you.  A  second  group  of  questions 
asks  how  you  feel  about  the  type  of  training  equipment  you  used  to  learn 
gasoline  engines  during  AIT  training.  A  third  group  of  questions  asks  how 
you  feel  overall  about  the  portion  of  the  training  program  that  covered 
gasoline  engines.  The  remaining  questions  ask  how  you  feel  about  your  AIT 
engine  training  in  general,  and  about  a  few  general  aspects  of  your  job. 

This  questionnaire  seems  long.  However,  almost  all  questions  can  be 
answered  by  either  making  a  circle  around  a  word  or  number,  or  placing  an  X 
in  a  blank.  There  are  only  very  few  questions  which  ask  you  to  write 
answers,  and  then  very  brief  answers  are  all  that  are  necessary. 

Please  consider  carefully  and  answer  all  questions  that  apply  to  you. 
In  the  few  cases  where  written  answers  are  asked  for,  please  take  the 
trouble  to  write  them.  Your  considered  judgments  and  thoughtful  answers 
are  needed  for  the  improvement  of  the  Army's  training  of  maintenance 
personnel. 


Trainee's  name: 
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I.  T'le  purpose  of  this  part  of  the  ques  t  ionn  t  ire  is  to  determine  how  well 
prepared  you  are  at  this  time  to  do  the  tasks  required  for  engine  main¬ 
tenance.  Lifted  below  are  a  number  of  these  '.asks .  For  each  task, 
please  circlt  the  number  on  the  right  under  the  coluim  head  that  you 
feel  best  de;  -ribes  your  ability  to  perform  each  tast  .  The  column 
heads  are  ex]  ained  below: 

1.  Not  prepared :  means  that  you  have  either  ni  t  learned  how 

to  do  this  task,  or  have  not  learned  it  wel  enough  to  try  it. 

2.  Need  close  supervision:  means  that  you  can  do  this  task,  hut 
only  with  close  supervision. 

3.  Need  minimum  supervision:  means  that  you  can  do  this  task 
fairly  well,  but  you  need  to  consult  with  someone  else  at 
some  point  for  directions  or  for  checking  results. 

4.  Need  no  supervision:  means  that  you  can  complete  tie  task 
with  3ut  help. 
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(Separati'  engine  maintenance  and  12  3  4 

repair  tasks  are  to  be  listed  here. 

Examples  are:  Adjusting  a  carburetor; 

Disassembl  ing/reass  ‘mbl ing  a  carburetor; 

Using  a  timing  light.) 


II.  The  purpose  of  this  part  of  the  questionnaire  is  to  determine  how 
much  you  have  improved  in  task  performance  since  assignment  to  your 
unit.  The  tasks  listed  below  are  the  same  as  shown  n  Part  I. 
above.  This  time,  however,  you  are  to  describe  how  you  have 


l^This  part  would  appear 
unit  assignment.  It  would  be 


only  in  questionnaires  administered  after 
omitted  in  the  one  givi  n  at  the  end  of  AIT. 
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changed  in  your  ability  to  do  the  tasks  since  you  were  assigned  to  this 
unit.  For  each  task,  please  circle  the  number  on  the  right  that  nest 
describe:  your  improvement.  If  you  needed  little  or  no  improvement  i  per¬ 
forming  a  given  task,  that  is,  if  you  could  do  the  task  when  first  assigned 
about  as  well  as  it  can  be  done,  please  place  an  X  in  the  column  headed  "No 
imprav'ement  needed."  Also,  circle  a  number  for  "Less  able"  or  "Equally 
able"  for  this  task  to  indicate  whether  you  have  maintained  this  skill.  The 
other  column  headings  are  explained  below: 

1.  Less  able:  means  that,  because  of  lack  of  practice  or  for 
any  other  reason,  you  cannot  do  this  task  now  as  well  as  you 
could  when  you  were  first  assigned  to  this  unit. 

2.  Equally  able:  means  that  you  can  do  this  task  as  well  (no 
better,  no  worse)  as  when  you  were  first  assigned  to  this  unit. 

3.  Some  improvement:  means  that  you  can  do  this  task  better 
than  you  could  when  you  wire  first  assigned  to  this  unit.  For 
example,  you  can  do  the  task  more  rapidly,  more  efficiently, 
with  fewer  errors,  etc. 

4.  Considerable  improvement:  means  that  you  can  do  this  task 
considerably  better  than  you  could  when  you  were  first 
assigned  to  this  unit 
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listed  in  Part  I.) 
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III.  The  two  parts  i./  of  the  questionnaire  above  were  concerned  with  your 
ability  to  do  actual  hands-on  tasks  with  engines.  The  purpose  of 
this  part  of  the  questionnaire  is  to  determine  how  much  you  Know  about 
how  engines  work  and  what  can  go  wrong  with  them.  Listed  below  are 
several  kinds  of  knowledge.  For  each  subject  or  kind,  please  circle 
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ee  footnote  for  Part  II. 
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the  number  under  the  column  head  that  you  feel  beat  describes  your 
knowledge  of  that  subject.  The  column  heads  are  explained  below. 

1.  Little  or  no  knowledge:  means  that  you  know  so  little  about 
this  subject  that  you  are  not  even  sure  what  all  tie  words 
mean  that  are  used  to  talk  about  it. 

2.  Some  knowledge :  means  that  you  know  enough  about  this  subject 
to  understand  generally  what  may  be  said  about  it,  but  you 
cannot  explain  it  to  someone  else  or  answer  many  questions 
about  it. 

3.  Quite  a  bit  of  knowledge:  means  that  you  know  enough  about 
this  subject  to  understand  fully  what  may  be  said  about  it, 
and  can  also  explain  much  of  it  to  someone  else  and  ar  ^er  co¬ 
worker's  usual  questions  about  it. 

4 .  Full  knowledge:  means  that  you  can  explain  this  topic  fully 
to  someone  else  and  answer  all  questions  about  it  that  a  co¬ 
worker  is  likely  to  ask. 


K  ,nd  of  knowledge 


(  Separate  aspects  of  engine  theory  and  1234 

f  mctioning  are  to  be  listed  here. 

Examples  are:  Valve  operations  during 
stroke  cycles;  How  the  carburetor  mixes 
fuel  and  air;  Causes  of  bearing  wear; 

Symptoms  of  incorrect  timing.) 

IV.  XJ  The  purpose  of  this  part  of  the  questionnaire  is  to  determine  how 
much  you  have  improved  in  your  knowledge  of  how  engines  worl;  and  what 
can  go  wrong  with  them  since  your  assignment  to  your  unit.  The  kinds 
of  knowledge  listed  below  are  the  same  as  those  in  Part  III.  above. 


i/Th  is  part  would  appear  only  in  questionnaires  administered  after 
unit  assignment.  It  would  be  omitted  in  the  one  given  at  the  end  of  AIT. 
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Full  knowledge 


This  time,  however,  you  are  to  describe  how  you  have  changed  in  vour 
knowledge  about  engines  since  you  were  assigned  to  your  unit.  For  each 
kind  of  knowledge,  please  circle  the  number  on  the  right  that  best 
describes  your  improvement.  If  you  needed  little  or  no  improvement  in 
knowledge  when  you  were  first  assigned  to  your  unit,  that  is,  if  you  knew 
about  as  much  as  you  needed  to  know  about  the  subject,  please  place  an  X 
in  the  column  headed  "No  improvement  needed."  Also,  circle  the  number 
for  "Know  less"  or  "Equally  able"  for  that  kind  of  knowledge  to  indicate 
whether  you  remember  this  knowledge.  The  other  column  headings  are 
explained  below. 

1.  Know  less:  means  that  you  have  forgotten  more  than  you  have 
learned  about  this  kind  of  knowledge  since  you  were  first 
assigned  to  this  unit. 

2.  No  change:  means  that  you  know  as  much  (no  more,  no  less)  about 
this  subject  now  as  when  you  were  first  assigned  to  this  unit. 

3.  Some  improvement:  means  that  you  now  know  somewhat  more  about 
this  subject  than  you  did  ^rfien  you  were  first  assigned  to  this 
unit . 

4.  Considerable  improvement:  means  that  you  now  know  a  great 
deal  more  about  this  subject  than  you  did  when  you  were  first 
assigned  to  this  unit. 


Kind  of  knowledge 


u 

C 

(U 

B 

0) 

> 

0) 

01 

U 

C 

d 

(U 

rH 

cd 

o 

CO 

bO 

B 

u 

u 

0) 

G 

o 

a) 

D-  'd 

»H 

n) 

> 

'd 

B  0) 

o 

•H 

M 

o 

d  u 

QO 

0) 

O 

B  Cu 

d 

o  a> 

o 

O  B 

0 

z 

C/5 

o 

c 

0) 

0 

(U 

> 

o 


(Kinds  of  knowledge  will 

be  the  same  as  in  Part  III.)  1234 

12  3  4 
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3 
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V.-A.  i./  The  purpose  of  this  part  of  the  questionnaire  is  to  determine 
how  yc u  feel  about  the  training  equipment  used  for  your  AIT  training  on 
gasoline  engines.  In  your  AIT  course  on  engines,  you  were  taught  using 
(to  be  completed  for  each  type  of  RPF  device).  Please  answer  the 
questions  below  about  this  device.  How  you  are  to  answer  each  question 
is  explained. 

How  useful  was  the  (device)  for  training  you  about  engines? 

(Check  one.) 

_ It  helped  me  a  lot 

_ It  helped  me,  but  I  could  have  learned  more  with  a  real  engine 

_ It  did  nt)t  help  me  much  at  all 

If  the  (device)  helped,  how  did  it  help  you?  (Check  as  many  as 
apply. ) 

_ It  did  not  help  me  much  at  all 

_ It  helped  me  understand  how  engines  work 

_ It  helped  me  understand  what  I  must  do  to  upair  and  maintain 

engines 

What  wa^  the  (device)  best  for  in  helping  you  learn?  (Write  in  your 
answer. ) 


What  was  taught  using  the  (device)  that  you  feel  should  have  been 
taught  some  other  way?  (Write  in  your  answer.) 


When  you  were  using  the  (device) ,  did  you  feel  that  you: 

(Circle 

Understood  how  to  use  the  (device)  to  Yes  No 

learn  what  you  were  supposed  to  learn? 

Understood  what  was  being  taught  with  it?  Yes  No  Jncertain 


i_/This  part  will  be  used  only  for  those  trainees  who  were  taught  in 
AIT  using  an  RPF  device  for  engine  training. 


one ) 

Jncertain 
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Were  helped  in  reaching  course  objectives? 

Yes 

No 

Uncertai n 

Were  learning  fast  enough? 

Yes 

No 

Uncertain 

Had  enough  information  about  how  well 
you  were  doing? 

Yes 

No 

Uncertain 

Were  making  good  use  of  your  time? 

Yef 

No 

Uncertain 

Would  you  prefer  that  most  or  all  of  the  time 
you  spent  using  a  (device)  be  spent  instead 
on  a  real  engine? 

Yes 

No 

Uncertain 

If  you  were  just  now  starting  the  AIT  course 
you  went  through,  would  you  want  to  use  this 
(device)  again? 

Yes 

No 

Uncertain 

If  Yes  or  Uncertain,  what  changes  should  be  made?  (Write  in  your 
answer. ) 


V. -B.  i./  The  purpose  of  this  part  of  the  questionnaire  is  to  determine  how 
you  feel  about  the  equipment  used  for  AIT  training  on  gasoline  engines. 

In  your  AIT  course  on  engines,  you  were  taught  using  an  engine  (to  be 
completed,  describing  how  engine  was  mounted,  etc.)  Please  answer  the 
questions  below.  How  you  are  to  answer  each  question  is  explained. 

How  useful  was  the  engine  for  training  you  about  engines? 

(Check  one.) 

_ It  helped  me  a  lot 

_ It  helped  me,  but  I  could  have  learned  more  using  something  else 

_ It  did  not  help  me  much  at  all 

If  the  engine  helped,  how  did  it  help  you?  (Check  as  many  as  apply.) 

_ ^It  did  not  help  me  much  at  all 

_ It  helped  me  understand  how  engines  work 

_ It  helped  me  understand  what  I  must  do  to  repair  and  maintain 

engines 

_ It  helped  me  learn  to  do  actual  "hands-on"  work  with  engines 


i/lhis  part  will  be  used  only  for  those  trainees  who  will  be 
taught  in  AIT  using  an  actual  engine  for  training. 
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What  was  the  engine  best  for  in  helping  you  learn?  (Write  in  your  answer.) 


What  was  taught  using  the  engine  that  you  feel  should  have  been  taught  some 
other  way?  (V/rite  in  your  answer.) _ _ 


When  you  were  using  the  engine,  did  you  feel  that  you: 


Understood  how  to  use  the  engine  to 
learn  wiiat  you  were  supposed  to  learn? 

Understood  what  was  being  taught  with 
it? 

Were  helped  in  reaching  course 
objectives  ? 

Were  learning  fast  enough? 

Had  enough  information  about  how  well 
you  were  doing? 

Were  making  good  use  of  your  time? 

Would  you  prefer  that  most  or  all  of  the 
time  you  spent  using  an  engine  be  spent 
instead  on  other  kinds  of  equipment? 

If  you  were  just  now  starting  the  AIT 
course  you  went  through,  would  you  want 
to  use  an  engine  like  this  again? 

Would  you  want  to  change  the  way  the 
engine  was  used  in  teaching? 


(Circle 
Yes  No 

one) 

Uncertain 

Yes 

No 

Uncertain 

Yes 

No 

Uncertain 

Yes 

No 

Uncertain 

Yes 

No 

Uncertain 

Yes 

No 

Uncertain 

Yes 

No 

Unc  irtain 

Yes  No  Uncertain 


Yes  No  line  irtain 


If  Yes  or  Uncertain,  what  changes  should  be  made?  (Write  in  your  answer.) 


VI. -A.  1/  In  AIT  you  went  through  what  is  called  a  lock-step  training  program 
on  gasoline  engines  where  you  and  your  classmates  usually  studied  everything 
together,  and  all  students  started  and  ended  each  part  at  the  same  time. 

In  some  programs,  each  student  is  allowed  to  advance  at  his  own  rate. 


jL^This  part  of  the  questionnaire  will  be  used  only  for  those  trainees  who 
were  taught  about  engines  in  a  lock-step  program. 
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The  purpose  of  this  part  of  the  questionnaire  is  to  determine  how  you  feel 
about  your  lock-step  program. 

In  general,  was  the  pace  of  the  engine  part  of  the  course  (check  one  blank) 

_ Too  fast  for  you? 

_ Too  slow  for  you? 

_ About  right  for  you? 

Were  there  some  topics  you  would  have  liked  more  time  on?  (Check  one  blank.) 

_ Yes 

_ No 

_ ^Uncertain 

If  your  answer  to  the  last  item  was  Yes  or  Uncertain,  what  topics  might  have 
needed  more  time?  (Write  in  your  answer.) _ 


Were  there  some  topics  which  too  much  time  was  spent  on?  (Check  one  blank.) 

_ Yes 

_ No 

_ Uncertain 

It  your  last  answer  was  Yes  or  Uncertain,  what  topics  really  needed  less 
time?  (Write  your  answer.) _ 


Did  you  feel  that  you  learned  as  much  as  you  should  have  about: 

(Circle  one) 


How  engines  work 

Yes 

No 

Uncet  tain 

How  you  are  to  go  about  repairing 
or  maintaining  an  engine 

Yes 

No 

Uncertain 

How  you  can  do  the  actual  "hands-on" 
jobs  you  will  need  to  do? 

Yes 

No 

Uncertain 

Did 

you 

your  AIT  program  give  you  all  the  time 
needed  to  learn  what  you  were  supposed 

Yes 

No 

Uncertain 

to  learn  about  engines? 
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VI. -B.  In  AIT  you  went  through  what  is  called  self-pacec'  training  on 

gasoline  engines  where  you  and  other  students  advanced  at  your  own  rates. 

In  some  programs  all  students  usually  studied  everything  together,  and  all 
students  started  and  ended  each  part  at  the  same  time.  The  purpose  of  this 
part  of  the  questionnaire  is  to  determine  how  you  feel  about  your  self-paced 
program. 

In  general,  was  the  pace  of  the  engine  part  of  the  course:  (check  one  blank) 
_ ^Too  fast  for  you? 


_ Too  slow  for  you? 

_ About  right  for  you? 

Were  there  some  topics  you  would  have  liked  more  time  on?  (Check  one  blank.) 

_ Yes 

_ No 

Uncertain 


If  your  answer  to  the  last  item  was  Yes  or  Uncertain,  what  topics  might  have 
needed  more  time?  (Write  in  your  answer.) _ _ 


Were  there  some  topics  which  too  much  time  was  spent  on?  (Check  one  blank.) 

_ ^Yes 

_ No 

_ Uncertain 

If  your  last  answer  was  Yes  or  Uncertain,  what  topics  really  needed  less 
time?  (Write  your  answer.) _ _ _ 


you  feel  that  you  learned  as  much  as  you 

should 

have 

about : 

(Circle 

one ) 

How  engines  work? 

Yes 

No 

Uncertain 

How  you  are  to  go  about  repairing 
or  maintaining  an  engine? 

Yes 

No 

Uncertain 

How  you  can  do  the  actual  "hands-on" 
jobs  you  will  need  to  do? 

Yes 

No 

Unc  'rtain 

i/This  part  of  the  questionnaire  will  be  used  o,.ly 
were  taught  about  engines  in  a  self-paced  program. 


for  tl  ose  trainees  who 


Did  your  AIT  program  give  you  all  the 
time  you  needed  to  learn  what  you  were 
supposed  to  learn  about  engines? 


Yea  No  Uncertain 


VII.  The  purpose  of  this  part  of  the  questionnaire  is  to  give  you  an  oppor¬ 
tunity  to  express  your  feelings  about  your  AIT  training  in  general. 

Please  answer  each  question  as  it  applies  to  you. 

Do  you  feel  that  you  can  do  the  tasks  listed  in  Part  I.  above  as  well 
as  you  can  be  expected  to  do  them  a  this  point  in  your  career? 

(Check  one. ) 


No 


Uncertain 


If  your  answer  was  No  or  Uncertain,  which  tasks  do  you  feel  you  were 
not  prepared  for  as  well  as  you  should  have  been?  (Please  write  your 
answer,  referring  to  the  list  in  Part  I.  as  necessary.) 


What  changes  should  be  made  in  the  AIT  training  program  to  help  it  provide 
the  engine  training  necessary  at  this  stage?  (Place  an  X  in  the  blank 
beside  each  change  you  feel  should  be  made.) 

_ No  change  needed 


More  time  is  needed  for  AIT  training 


Less  time  is  needed  for  AIT  training 


More  equipment  hands-on  training  is  needed 


_ ^The  AIT  training  equipment  should  be  improved 

_ The  AIT  training  equipment  should  be  used  more 


More  theory  of  how  engines  operate  should  be  covered 
Less  theory  of  how  engines  operate  should  be  covered 


More  AIT  training  time  should  be  spent  on  how  to  do  maintenance  tasks 
Less  AIT  training  time  should  be  spent  on  how  to  do  maintenance  tasks 
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The  An  training  should  be  different  (different  in  what  way?) 


) 


AIT  training  should  go  deeper  into  some  topics  (which  topics?) 


} 


In  terms  of  your  ability  to  work  with  engines  that  you  should  have  at 
this  stage  in  your  career,  what  do  you  feel  you  are  especially  well 
prepared  to  do?  _ 


What  are  you  especially  weak  in  regarding  engines?  _ 

i 


VIII.  All  in  all,  how  valuable  was  the  trainin}^,  equipment  used  for  teaching  * 

you  about  engines  in  AIT?  (Check  one  blankT5 

_ Very  valuable  i 

_ Valuable,  but  it  could  have  been  better 

_ Of  some  value,  but  a  lot  of  improvement  is  needed  ^ 

Of  no  value 


Uncertain 


All  in  all,  how  valuable  was  your  AIT  training  on  engines? 
(Check  one  blank.) 

_ Very  valuable 


Valuable,  but  it  could  have  been  better 


Of  some  value,  but  a  lot  of  improvement  is  needed 


Of  some  value 


Uncertain 


[ilil 


Regardless  of  AIT  training,  what  is  your  overall  ini  {re  sion  of  your 
present  ability  to  maintain  and  repair  engines?  (Check  one  blank.) 

_ Very  well  qualified 

_ Adequately  qualified 

_ Barely  qualified 

_ Not  qualified 

How  much  confidence  do  you  have  in  your  ability  to  maintain  and  repair 
engines?  (Check  one.) 

_ feel  sure  I  can  do  a  good  job  on  almost  any  task 

_ I  feel  I  (.an  do  a  good  job  most  of  the  time 

_ I  feel  I  cannot  do  a  good  job  most  of  the  time 

_ I  feel  that  I  am  not  able  to  do  the  job  required  of  me 

How  do  you  feel  about  your  maintenance  job?  (Check  one.) 

_ 1  am  very  enthusiastic;  I  enjoy  my  job  very  much 

_ I  am  happy  enough  with  my  job,  but  sometimes  I  do  not  enjoy  it 

_ am  not  content  with  my  job,  but  I  make  the  best  of  it 

_ I  am  very  dissatisfied  with  my  job;  I  should  be  doing  something  else 

Please  use  the  space  below,  or  backs  of  pages  if  needed,  to  write 
comments  that  will  help  in  finding  out  how  valuable  your  AIT  training 
on  engines  was.  You  can  also  use  this  space  is  you  wish  to  comment 
on  the  questions  you  have  answered. 


2.  Specimen  Insti actor  Questionnaire  for  Evaluating  Training 
for  the  Maintenance  and  Repair  of  Gasoline  Engines 

The  lurpose  of  this  questionnaire  is  to  determine  how  you  feel  regar  ling  the 
ability  of  a  typical  graduating  AIT  trainee  to  maintain  and  repair  gasoline 
engines,  and  xegarding  his  AIT  training  to  maintain  and  r.^pair  engines. 

Five  types  of  information  are  covered.  First,  there  are  questions  asking 
how  well  prepared  you  feel  the  trainee  is  to  do  the  maintenance  and  repairs 
on  gasoline  engines  that  are  expected  of  him.  A  second  group  of  questions 
asks  bow  you  feel  about  his  knowledge  of  engines.  \  third  group  of 
questions  asks  you  to  evaluate  certain  aids  used  in  teaching  engines.  A 
fourth  group  asks  you  to  evaluate  certain  ways  of  taaching  engines. 

Finally,  the  remaining  questions  focus  on  AIT  training  in  general. 

This  questionnaire  seems  long.  However,  almost  all  questions  can  be 
answered  by  either  making  a  circle  around  a  word  or  number,  or  placing  an 
X  in  a  blank.  There  are  only  very  few  questions  which  ask  you  to  write 
answers,  and  then  very  brief  answers  are  all  that  are  necessary. 

Please  consider  carefully  and  answer  all  questions  about  which  you  have  suf¬ 
ficient  information.  In  the  few  cases  where  written  answers  are  asked  foe, 
please  take  the  trouble  to  write  them.  Your  considered  judgments  and 
thoughtful  answers  are  needed  for  the  improvement  of  the  Army's  training  of 
maintenance  personnel. 


Instructor’s  name: 
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I.  The  purpose  of  this  part  of  the  questionnaire  is  to  determine  how  well 
prepared  the  typical  trainee  is,  upon  completing  AIT,  to  d('  the  tasks 
required  for  engine  maintenance.  Listed  below  are  a  number  of  these 
tasks.  For  each  task,  please  circle  the  number  on  the  right  under  the 
column  head  that  you  feel  best  describes  the  trainee's  ability  to  per¬ 
form  each  task.  Tlie  column  heads  are  explained  below: 

1.  Not  prepr.red :  means  that  the  typical  graduating  trainee  either 
has  not  learned  how  to  do  this  task,  or  has  not  learned  it  well 
enough  to  try  it. 

2.  Needs  close  supervision:  means  that  the  typical  trainee  can  dc 
this  task,  but  only  with  close  supervision. 

3.  Needs  minimum  supervision:  means  that  the  typical  trainee  can 
do  this  task  fairly  well,  but  he  needs  to  consult  with  someone 
else  at  some  point  for  directions  or  for  checking  results. 

A.  Needs  no  supervision:  means  that  the  typical  trainee  can 
complete  the  task  without  help. 

Insufficient  information:  means  that  you  have  not  had  enough 
opportunity  to  observe  trainees  to  rate  their  performance  on  a 
given  task.  If  this  is  true  for  a  given  task,  place  an  X  in 
this  column  opposite  that  task. 


Task 
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(.Separate  engine  maintenance  and 
repair  tasks  are  to  be  listed  here. 
Examples  are:  Adjusting  a  carburetor; 
Disassembling/reassembling  a  carbure¬ 
tor;  Using  a  timing  light.) 


12  3  4 
12  3  4 
12  3  4 


II.  The  part  of  the  questionnaire  above  was  concerned  with  the  typical 
graduating  trainee's  ability  to  do  actual  hands-on  tasks  with  engines. 
The  purpose  of  this  part  of  the  questionnaii  t*  is  to  determine  how  much 
the  typical  trainee  knows  about  how  engines  work  and  what  can  go  wrong 
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Insufficient 

information 


with  them.  Listed  below  are  several  kinds  of  knowledge.  For  each  s  ib- 
ject  or  kind,  please  circle  ''he  number  under  the  column  head  that  yo  i 
feel  best  describes  the  typi.al  trainee's  knowledge  of  that  subject. 


The  column  h ?ads  are  explained  below. 


1.  Little  or  no  knowledge:  means  that  the  typical  graduating  trainee 
knows  so  little  about  this  subj<.  ;  that  he  is  not  even  sure  what 
all  the  words  mean  that  are  used  to  talk  about  it. 


2.  Some  knowledge:  means  that  the  typical  trainee  knows  enougl  about 
this  subject  to  understand  generally  what  may  be  said  about  it,  but 
he  cannot  explain  it  to  someone  else  or  answer  many  questions  about 
it. 

3.  Quite  a  bit  of  knowledge:  means  that  the  typical  trainee  knows 
enough  about  this  subject  to  understand  fully  what  may  be  said 
about  it,  and  can  also  explain  much  of  it  to  someone  else  and 
answer  co-workers'  usual  questions  about  it. 

A.  Full  knowledge:  means  that  the  typical  trainee  can  explain  this 
topic  fully  to  someone  else  and  answer  all  questiois  about  it  that 
a  co-worker  is  likely  to  ask. 

Insufficient  information;  means  that  you  have  not  had  enough 
opportunity  to  observe  trainees  to  rate  their  knowledge  on  this 
subject.  If  this  is  the  case  for  a  given  subject,  place  an  X  in 
this  colu.nn  opposite  that  subject. 


Kind  of  knowledge 
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(Separat  ?  aspects  of  engine  theory 
and  functioning  are  to  be  listed  here. 
Examples  are:  Valve  operations 
during  stroke  cycles;  How  the  car¬ 
buretor  mixes  fuel  and  air;  Causes  of 
bearing  wear;  Symptoms  of  incorrect 
t iming. ) 


12  3  4 
12  3  4 
12  3  4 
12  3  4 
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III.  W  The  purpose  of  this  part  of  the  questionnaire  is  to  determine  how 
you  feel  about  the  training  equipment  you  have  used  in  AIT  training  on 
gasoline  engines.  In  your  AIT  courses  on  gasoline  engines,  you  have 
used  (identification  of  an  RPF  to  be  inserted. ) 

Please  answer  the  questions  below  about  this  device.  How  you  are  to 
answer  each  question  is  explained. 

How  much  experience  have  you  had  with  this  teaching  device?  (Check 
one. ) 


I  have  used  it  only  once 

I  have  used  it  more  than  once,  but  no  more  than  three  times 
I  have  used  it  more  than  three  times 
How  useful  was  this  device  for  engine  training?  (Check  one.) 

_ ^It  helped  a  lot 

It  helped,  but  more  could  have  been  learned  using  a  real  engine 
_ It  did  not  help  much  at  all 

If  the  device  helped,  how  did  it  help?  (Check  as  many  as  apply.) 

_ ^It  did  not  help  much  at  all 

_ ^It  helped  teach  how  engines  work 

It  helped  teach  what  must  be  done  to  repair  and  maintain  engines 

_ ^It  helped  teach  actual  "hands-on"  work  with  engines 

What  was  the  device  best  for  in  helping  trainees  learn?  (Write  in  your 
answer. ) 


What  was  taught  using  the  device  that  you  feel  should  have  been  taught 
some  other  way?  (Write  in  your  answer.)  _ _ _ 


l/xhis  section  of  the  questionnaire  is  to  be  repeated 
device  used.  If  an  instructor  has  not  used  such  a  device, 
would  be  omitted  from  his  questionnaire. 


for  each  RPF 
this  section 
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A 


When  you  were  using  the  device,  did  you  feel  that  the  trainees: 


Understood  how  to  use  the  device  to 
learn  what  they  were  supposed  to 
learn? 

Yes 

(Circle 

No 

one) 

Uncertf in 

Understood  what  was  being  taught  with 
it? 

Yes 

No 

Uncert  tin 

Were  helped  in  reaching  course 
objectives? 

Yes 

No 

Uncertain 

Were  learning  fast  enough? 

Yes 

No 

Uncertain 

Had  enough  information  about  how  well 
they  were  doing? 

Yes 

No 

Uncertain 

Were  making  good  use  of  their  time? 

Yes 

No 

Uncertain 

When  you  were  using  the  device  did  you  feel  that  the  device : 


(Circle 

one) 

a. 

Was  easy  to  set  up  for  teaching? 

Yes 

No 

Uncertain 

b. 

Was  easy  for  an  instructor  to  use 
after  setup? 

Yes 

No 

Uncertain 

c. 

Was  easy  for  the  trainees  to  use? 

Yes 

No 

Uncertain 

d. 

Provided  for  evaluations  of  enough 
different  types  of  student 
performance? 

Yes 

No 

Uncertain 

e. 

Provided  rapid  enough  feedback  to 
trainees  regarding  how  they  were 
doing? 

Yes 

No 

Uncertain 

f. 

Had  the  software  needed  to  capi¬ 
talize  adequately  on  the  device's 
training  potential? 

Yes 

No 

Uncerta in 

g« 

Was  dependable  in  that  it  did  not 
require  an  undue  amount  of 
maintenance  and  repair? 

Yes 

No 

Uncertain 

h. 

Provided  for  teaching  a  good  range 
of  subject  matter? 

Yes 

No 

Uncertain 

i . 

Provided  ample  opportunity  for 
trainees  to  practice  without  undue 
supervsion  by  you? 

Yes 

No 

Uncertain 

j  • 

Had  sufficient  fidelity  to  actual 
engines  for  teaching  purposes? 

Yes 

No 

Uncertain 
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You  will  note  that  the  questions  just  preceding  regarding  device  charac¬ 
teristics  are  identified  by  letters  a.  through  j.  You  may  have  circled  No 
or  Uncertain  for  one  or  more  of  them.  If  so,  please  use  the  space  below  to 
state  why  you  did  not  feel,  or  were  uncertain,  that  this  characteristic  is 
satisfactory  for  this  device.  For  each  No  or  Uncertain  circled,  please 
identify  the  corresponding  letter  in  the  space  below  and  state  briefly  after 
the  letter  what  you  consider  needs  improvement  in  the  device.  If  more  space 
is  needed,  you  may  write  on  the  back  of  this  page. 


Would  you  want  to  change  the  way  this  Yes  No  Uncertain 

device  is  used  in  teaching? 

What  changes  should  be  made?  (Write  in  your  answer.) _ 


If  you  were  free  to  choose,  would  you  use  Yes  No  Uncertain 

this  device  in  all  similar  courses? 

If  you  circled  No  or  Uncertain,  what  would  you  prefer  instead  of  this 
device?  (Check  both  if  you  would  prefer  either  to  this  device.) 

_ A  real  engine 


Another  device  (which  one?) 
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IV.  i./  The  purpose  of  this  part  of  the  questionnaire  is  to  determine  how 
you  feel  about  some  actual  engine  equipment  you  have  used  in  AIT 
training  on  gasoline  engines.  In  your  AIT  courses  on  gasoline  engines, 
you  have  used  (identification  of  an  AET  configuration,  including 
definition  of  AET,  to  be  inserted). 

Please  answer  the  questions  below  about  this  AET.  How  you  are  to  answer 
each  question  is  explained. 

How  much  experience  have  you  had  with  this  AET?  (Check  one.) 

_ T  have  used  it  only  once 

_ have  used  it  more  than  once,  but  no  more  than  three  times 

_ I  have  used  it  more  than  three  times 

How  useful  was  this  AET  for  engine  training?  (Check  one.) 

_ It  helped  a  lot 

_ It  helped,  but  more  could  have  been  learned  using  another  type  of 

teaching  aid 

_ It  did  not  help  much  at  all 

If  the  AET  helped,  how  did  it  help?  (Check  as  many  as  apply.) 

_ It  did  not  help  much  at  all 

_ It  helped  teach  how  engines  work 

_ It  helped  teach  what  must  be  done  to  repair  and  maintain  engines 

_ It  helped  teach  actual  "hands-on"  work  with  engines 

What  was  the  AET  best  for  in  helping  trainees  learn?  (Write  in  your 
answer. ) 


What  was  taught  using  the  AET  that  you  feel  should  have  been  taught  some 
other  way.?  (Write  in  your  answer.) 


i_/Thi3  section  of  the  questionnaire  is  to  be  repeated  tor  each 
basically  different  AET  configuration  used.  If  an  instructor  has  not 
used  AETs,  this  section  would  be  omitted  from  his  questionn; ire . 
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;■  1^1  III 


'«  When  you  were  using  the  AET,  did  you  feel  that  the  trainees : 


(Circle 

one) 

Understood  how  to  use  the  AET  to  learn 
what  they  were  supposed  to  learn? 

Yes 

No 

Uncertain 

Understood  what  was  being  taught  with 
it? 

Yes 

No 

Uncertain 

Were  helped  in  reaching  course 
objectives? 

Yes 

No 

Uncertain 

Were  learning  fast  enough? 

Yes 

No 

Uncertain 

Had  enough  information  about  how 
well  they  were  doing? 

Yes 

No 

Uncertain 

Were  making  good  use  of  their  time? 

Yes 

No 

Uncertain 

When  you  were  using  the  AET  did  you  feel  that  the  AET: 

Tc^ircle  one) 


a. 

Was  easy  to  set  up  for  teaching? 

Yes 

No 

Uncertain 

b* 

Was  easy  for  the  instructor  to  use 
after  set-up? 

Yes 

IIo 

Uncertain 

c. 

Was  easy  for  the  trainees  to  use? 

Yes 

No 

Uncertain 

d. 

Provided  for  evaluations  of  enough 
different  types  of  student  per¬ 
formance? 

Yes 

No 

Uncertain 

e. 

Provided  rapid  enough  feedback  to 
trainees  regarding  how  they  were 
doing? 

Yes 

No 

Uncertain 

f. 

Was  dependable  in  that  it  did  not 
require  an  undue  amount  of 
maintenance  and  repair? 

Yes 

No 

Uncertain 

g- 

Provided  for  teaching  a  good  range 
of  subject  matter? 

Yes 

No 

Uncertain 

h . 

Provided  ample  opportunity  for 
trainees  to  practice  without  undue 
supervision  by  you? 

Yes 

No 

Uncertain 

'  i . 

Had  sufficient  fidelity  to 

Yes 

No 

Uncertain 

functional  engines  in  vehicles  for 
teaching  purposes? 


1 
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You  will  note  that  the  questions  just  preceding  regarding  AET  characteristics 
are  identified  by  letters  a.  through  i.  You  may  have  circled  No  or 
Uncertain  for  one  or  more  of  them.  If  so,  please  use  the  space  below  to 
state  why  you  did  not  feel,  or  were  uncertain,  that  this  characteristic  is 
satisfactory  for  this  AET.  For  each  No  or  Uncertain  circled,  please  iden¬ 
tify  the  corresponding  letter  in  the  space  below  and  state  briefly  after  the 
letter  what  you  consider  needs  improvement  in  this  AET.  If  more  space  is 
needed,  you  may  write  on  the  back  of  this  page. 


Would  you  want  to  change  the  way  Yes  No  Uncertain 

this  AET  is  used  in  teaching? 


What  changes  should  be  made:  (Write  in  your  answer.) 


If  you  were  free  to  choose,  would  you  Yes  No  Uncertain 

use  this  AET  in  all  similar  courses? 

If  you  circled  No  or  Uncertain,  what  would  you  prefer  instead  of  this  AET? 
(Check  both  if  you  would  prefer  either  to  this  AET.) 

_ A  different  AET  (which  one?) 

_ An  engine  simulator  designed  for  teaching  (\/hich  one?) 


V.  1.1  In  AIT  you  taught  in  %rtiat  ia  called  a  lock-step  training  program 
on  gasoline  engines  where  trainees  usually  studied  everything  together, 
and  all  students  started  and  ended  each  part  ut  the  seme  time.  (In  some 
programs,  each  student  is  allowed  to  advance  at  his  own  rate.)  The  purpose 
of  this  part  of  the  questionnaire  is  to  determine  how  you  feel  about  the 
lock-step  program. 

In  general,  was  the  pace  of  the  engine  part  of  the  course 
(check  one  blank) 


_ Too  fast  for  the  typical  trainee? 

_ Too  slow  for  the  typical  trainee? 

_ About  right  for  the  typical  trainee? 

Were  there  some  topics  you  would  have  liked  to  spend  more  time  on? 
(Check  one  blank.) 


_ Uncertain 

If  your  answer  to  the  last  item  was  Yes  or  Uncertain,  what  topics  might 
have  needed  more  time?  (Write  in  your  answer.) _ 


Were  there  some  topics  which  too  much  time  was  spent  on?  (Check  one  blank.) 


_ ^Uncertain 

If  your  last  answer  was  Yes  or  Uncertain,  what  topics  really  needed  less 
time?  (Write  your  answer.) _ 


aire  will  be  used  on 
a  lock-step  program 


Did  you  feel  that  the  typical  trainee  learned  as  much  as  he  should  have 
about : 

(Circle  one) 


How  engines  work? 

Yes 

No 

Uncertain 

How  he  is  to  go  about  repairing 
or  maintaining  an  engine? 

Yes 

No 

Uncertain 

How  he  can  do  the  actual  "hands-on" 
jobs  he  will  need  to  do? 

Yes 

No 

Uncertain 

Did 

the 

the  lock-step  AIT  program  give  him  all 
time  he  needed  to  learn  what  he  was 

Yes 

No 

Uncertain 

supposed  to  learn  about  engines? 


VI.  In  AIT  you  taught  in  what  is  called  a  self-paced  training  program 

on  gasoline  engines  where  students  advanced  at  their  own  rates.  (In  some 
other  programs  all  students  usually  studied  everything  together,  and  all 
students  started  and  ended  each  part  at  the  same  time.)  The  purpose  of 
this  part  of  the  questionnaire  is  to  determine  how  you  feel  about  the 
sel f-paced  program. 

In  ’eneral,  was  the  pace  of  the  engine  part  of  the  course  (check  one  alank) 

_ Too  fast  for  the  typical  trainee? 

_ Too  slow  for  the  typical  trainee? 

_ About  right  for  the  typical  trainee? 

Were  there  some  topics  you  would  have  liked  to  spend  more  time  on?  (Check 
one  blank.) 

_ Yes 

No 


Uncertain 


If  your  answer  to  the  last  item  was  Yes  or  Uncertain,  what  topics  might 
have  needed  more  time?  (Write  in  your  answer.) _ 


i.^'This  part  of  the  questionnaire  will  be  used  only  for 
instructors  who  taught  engines  in  a  self-paced  program. 


those 
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Were  there  some  topics  which  too  much  time  was  spent  on?  (Check  one  blank.) 


_ Yes 

_ No 

_ ^Uncertain 

If  your  last  answer  was  Yes  or  Uncertain,  what  topics  really  needed  less 
time?  (Write  your  answer.) _ 


hid  you  feel  that  the  typical  trainee  learned  as  much  as  he  should  have 
about : 

(Circle  one) 


How  engines  work? 

Yes 

No 

Unc 'rtain 

How  he  is  to  go  about  repairing 
or  maintaining  an  engine? 

Yes 

No 

Uncertain 

How  he  can  do  the 
jobs  he  will  need 

actual  "hands-on" 
to  do? 

Yes 

No 

Uncertain 

the  self-paced  AIT 

program  give  him  all  the 

Yes 

No 

Uncertain 

time  he  needed  to  learn  what  he  was  supposed  to 
learn  about  engines? 

VII,  The  purpose  of  this  part  of  the  questionnaire  is  to  give  you  an  oppor¬ 
tunity  to  express  your  feelings  about  AIT  training  in  general.  Please 
answer  each  question  as  it  applies  to  you. 

Do  you  feel  that  the  typical  graduating  trainee  can  do  the  tasks 
listed  in  Part  I.  above  as  well  as  he  can  be  expected  to  do  them  at 
this  point  in  his  career?  (Check  one.l 

Yes 


No 


Uncertain 


If  your  answer  was  No  or  Uncertain,  which  tas..8  do  you  feel  the  typi¬ 
cal  trainee  is  not  prepared  for  as  well  as  he  should  be.  (Please 
write  your  answer,  referring  to  the  list  in  Part  I.  as  necessary.) 
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What  changes  should  be  made  in  the  AIT  training  program  to  help  it 
provide  the  engine  training  necessary  at  this  stage?  tPlace  an  X  in 
the  blank  beside  each  change  you  feel  should  be  made.) 

_ No  changes  needed 

_ More  time  is  needed  for  AIT  training 

_ Less  time  is  needed  for  AIT  training 

_ More  equipment  hands-on  training  is  needed 

_ The  AIT  training  equipment  should  be  improved 

_ The  AIT  training  equipment  should  be  used  more 

_ More  theory  of  how  engines  operate  should  bo  covered 

_ Less  theory  of  how  engines  operate  should  be  covered 

_ More  AIT  training  time  should  be  spent  on  how  to  do 

maintenance  tasks 

_ Less  AIT  training  time  should  be  spent  on  how  to  do 

maintenance  tasks 

_ The  AIT  training  should  be  different  (different  in  what  way?) 


What  is  he  especially  weak  in  regarding  engines? 


VIII.  All  in  all,  how  valuable  was  the  training  equipment  u;ied  for 
teaching  about  engines  in  AIT?  (Check  one  blank.) 

_ Very  valuable 

_ Valuable,  but  it  could  have  been  better 
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_ ^Of  some  value,  but  a  lot  of  improvement  is  needed 

_  _0f  no  value 

_ Uncertain 

All  in  all,  how  valuable  is  AIT  training  on  engines?  (Check  one 
blank. ) 


Very  valuable 

Valuable,  but  it  could  have  been  better 


Of  some  value,  but  a  lot  of  improvement  is  needed 


Of  no  value 


Uncertain 


Regardless  of  AIT  training,  what  in  your  overall  impression  of 
a  graduating  trainee's  ability  to  aintain  and  repair  engines? 

(Check  one  blank.) 

_ Very  well  qualified 

_ Adequately  qvialified 

_ ^Barely  qualified 

_ jNot  qualified 

Whether  or  not  the  typical  trainee  is  realistic,  how  much  con¬ 
fidence  does  he  have  in  his  ability  to  maintain  and  repair 
engines?  (Check  one.) 

_ Feels  sure  he  can  do  a  good  job  on  almost  any  task 

_ Feels  he  can  do  a  good  job  most  of  the  time 

_ Feels  he  cannot  do  a  good  job  most  of  the  time 

_ Feels  that  he  is  not  able  to  do  the  job  required  of  him 

Are  the  typical  trainee's  feelings  about  his  ability  realistic? 

That  is,  does  he  show  overconfidence,  too  little  confidence,  or  just  the 
right  amount?  (Check  one.) 

_ ^Shows  overconfidence;  he  is  not  as  good  as  he  thinks  he  is 

_ ^Shows  too  little  confidence;  he  is  better  than  he  thinks  he  is 

_ He  is  realistic  in  his  feelings  about  his  ability 
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How  does  the  typicil  trainee  feel  about  his  job?  (Cl  eck  oni  .) 

_ _See.Ti8  ve -y  enthusiastic;  seems  to  enjoy  his  job  V'  ry  much 

_ Seems  haupy  enough  with  his  job,  but  sometimes  set-ms  not 

to  enjoy  it 

_ Seems  di  icontented  with  his  job,  but  makes  the  best  of  it 

_ ^Seems  very  dissatisfied  with  his  job;  he  should  be  doing 

something  else 

Please  use  the  space  below,  or  backs  of  pages  if  needd,  to  write 
comments  that  will  help  in  finding  out  how  valuable  AT  training 
on  engines  is.  You  can  also  use  this  space  if  you  with  to  comment 
on  the  questions  you  have  answered. 
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3.  Specimen  Supervisor  Questionnaire  for  Evaluating  Training 
for  the  Maintenance  and  Repair  of  Gasoline  Engines 


This  questionnaire  concerns  the  mechanic  identified  below.  The  pur¬ 
pose  of  the  questionnaire  is  to  determine  how  you  feel  regarding  his 
ability  to  maintain  and  repair  engines.  Three  types  of  information  are 
covered.  First,  there  are  questions  asking  how  well  prepared  you  feel  the 
mechanic  is  to  do  the  raintenauce  and  repairs  on  gasoline  engines  that  are 
expected  of  him.  A  second  group  of  questions  asks  how  you  feel  about  his 
knowledge  of  engines.  The  remaining  questions  focus  on  the  mechanic's 
AIT  training  in  general  and  a  few  general  aspects  of  his  job  performance. 

This  questionnaire  seems  long.  However,  almost  all  questions  can  be 
answered  by  either  making  a  circle  around  a  word  or  number,  or  placing  an 
X  in  a  blank.  There  are  only  very  few  questions  which  ask  you  to  trrite 
answers,  and  then  very  brief  answers  are  all  that  are  necessary. 

Please  consider  carefully  and  answer  all  questions  about  which  you 
have  sufficient  information.  In  the  few  cases  where  written  answers  are 
asked  for,  please  take  the  trouble  to  write  them.  Your  considered 
judgments  and  thoughtful  answers  are  needed  for  the  improvement  of  the 
Army's  training  of  maintenance  personnel. 


Mechanic  being  rated: 


Rater's  name: 
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I.  The  purnoce  of  this  part  of  the  questionnaire  is  to  determine  how  well 
prepared  tl  ^  mechanic  is  at  this  time  to  do  the  tasks  required  for  engine 
maintenance.  Listed  below  are  a  number  of  these  taslcs.  For  each  task, 
please  circle  the  number  on  the  right  under  the  coluim  head  that  you  feel 
best  describes  the  mechanic's  ability  to  perform  eacl  task.  The  column 
heads  are  explained  below: 

1.  Not  prepared:  means  that  the  mechanic  has  either  rot  learned  how 
to  do  this  task,  or  has  not  learned  it  well  enough  to  try  it. 

2.  Needs  close  supervision:  means  that  the  mei  hanic  can  do  this 
task,  but  only  with  close  supervision. 

3.  Needs  minimum  supervision:  means  that  the  mechanic  can  do  this 
task  fairly  well,  but  he  needs  to  consult  with  someone  else  at 
some  point  for  directions  or  for  checking  results. 

4.  Needs  no  supervision:  means  that  the  mechanic  can  complete  the 
task  withojt  help. 

Insufficient  information:  means  that  you  have  not  had  enough 
opportunity  to  observe  the  mechanic  to  rate  his  performance  on  a 
given  task.  If  this  is  true  for  a  given  task,  place  an  X  in  this 
column  opposite  that  task. 


Task 


T3 

3 

0) 

0) 

C 

B 

0 

d 

U 

(A 

o 

•H 

o 

o 

CTJ 

O 

•H 

C 

•rt 

•H 

a 

CO 

CO 

o 

CO 

0) 

o 

•H 

c 

•r4 

u 

> 

> 

> 

a 

U 

CO 

U 

CO 

u 

0) 

'V 

0) 

u 

0) 

a 

0) 

a 

0) 

cx 

o 

Q) 

0) 

0 

0) 

3 

z 

7U 

Z 

CO 

z 

CO 

(Separate  engine  maintenance  and 
repair  tasks  are  to  be  listed  here. 
Examples  are:  Adjusting  a  carburetor; 
Disassembling/reassembling  a  carburetor; 
Using  a  timing  light.) 


12  3  4 
12  3  4 
12  3  4 


II.  The  purpose  of  this  part  of  the  questionnaire  is  to  determine  how  much 
the  mechanic  has  improved  in  task  performance  since  his  assignment  to 
your  unit.  The  tasks  listed  below  are  the  same  as  those  in  Part  I. 
above.  This  time,  however,  you  are  to  describe  how  the  mechanic  has 
changed  in  ability  to  do  the  tasks  since  he  has  been  assigned  to  this 
unit.  For  each  task,  please  circle  the  number  on  the  right  that  best 
describes  his  improvement.  In  the  event  that  the  mechanic  needed 
little  or  no  improvement  in  performing  a  given  task,  that  is,  he  could 
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Insufficient 

information 


A  ' 


do  the  task  at  the  outset  about  as  well  as  it  can  be  done,  please 
place  an  X  in  the  column  headed  "No  improvement  needed."  AIjo,  circle  a 
number  for  "Less  able"  or  "Equally  able"  for  this  task  to  indicate 
whether  the  mechanic  has  maintained  this  sKill.  The  ocher  column 
headings  are  explained  below; 


1.  Less  able:  means  that,  because  of  lack  of  practice  or  for  any 
other  reason,  the  mechanic  cannot  do  this  task  now  as  well  as 
he  could  when  he  was  first  assigned  to  this  unit. 

2.  Equally  able:  means  that  the  mechanic  can  do  this  task  as  well 
(no  better,  no  worse)  as  when  he  was  first  assigned  to  this  unit. 

3.  Some  improvement:  means  that  the  mechanic  can  do  this  task 
better  than  he  could  when  he  was  first  assigned  to  this  unit. 

For  example,  he  can  do  the  task  more  rapidly,  more  efficiently, 
with  fewer  errors,  etc. 

4.  Considerable  improvement:  means  that  the  mechanic  can  do  this 
task  considerably  better  than  he  could  when  he  was  first  assigned 
to  this  unit. 

Insufficient  information:  means  that  you  have  not  had  enough 
opportunity  to  observe  the  mechanic  to  rate  his  performance  on 
a  given  task.  If  this  is  true  for  a  given  task,  place  an  X  in 
this  column  opposite  that  task. 


Task 


u 

C 

4J 

0) 

c 

B 

0) 

(P 

0) 

0) 

u 

B 

iH 

> 
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U 

d 

d 

Q> 

X 

0 

X 

d 

a> 

o 

> 
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u 

0 

■H 
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o 

fH 

a 

U 

B 

O 

U 

u 

Xi 

>> 

B 

Q) 

Q> 

•H 

d 

a  TJ 

n) 

•H 

'O 

> 

4-1 

Q 

B  0) 

iH 

•H 

O 

M-i 

M 

*H  "0 

UO 

0) 

U 

3 

0 

a> 

CO 

■3 

5 

d 

Q* 

00 

4^ 

C  0) 

(U 

U* 

0 

D 

B 

d 

d 

as  c 

W 

CJ 

1-4 

(Tasks  will  be  identical  to  those 
listed  in  Part  I.) 


12  3  4 
12  3  4 
12  3  4 


III.  The  two  parts  of  the  questionnaire  above  were  concerned  with  the 
mechanic's  ability  to  do  actual  hands-on  tasks  with  engines.  The  pur¬ 
pose  of  this  part  of  the  questionnaire  is  to  determine  how  much  the 
mechanic  knows  about  how  engines  work  and  what  can  go  wrong  with 
them.  Listed  below  are  several  kinds  of  knowledge.  For  each  subject 
or  kind,  please  circle  the  number  under  the  column  head  that  you  feel 
best  describes  this  mechanic's  knowledge  of  that  subject.  The  column 
heads  are  explained  below. 
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1.  Little  or  no  knowledge:  means  that  the  mechanic  knows  so  little 
about  this  subject  that  he  is  not  even  sure  what  all  the  words  mean 
that  are  used  to  talk  about  it. 

2.  Some  knowledge:  means  that  the  mechanic  knows  enoug  i  about  this 
subject  to  understand  generally  what  may  be  said  about  it,  but  he 
cannot  explain  it  to  someone  else  or  answer  many  que-tions  about 
it . 

3.  Quite  a  bit  of  knowledge:  means  that  the  mechanic  knows  enough 
about  this  subject  to  understand  fully  what  may  be  said  about  it, 
and  can  also  explain  much  of  it  to  someone  else  and  answer  co¬ 
workers'  usual  questions  about  it. 

4.  Full  knowledge:  means  that  the  mechanic  can  explain  this  topic 
fully  to  someone  else  and  answer  all  questions  about  it  that  a  co¬ 
worker  is  likely  to  ask. 

Insufficient  information:  means  that  you  have  not  had  enough 
opportunity  to  observe  the  mechanic  to  rate  his  knowledge  on  this 
subject.  If  this  is  the  case  for  a  given  subject,  place  an  X  in 
this  column  opposite  that  subject. 
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(Separate  aspects  of  engine  theory  1234 

and  functioning  are  to  be  listed 

here.  Examples  are:  Valve  operations  1234 

during  stroke  cycles;  How  the  carburetor 

mixes  fuel  and  air;  Causes  of  bearing  1234 

wear;  Symptoms  of  incorrect  timing.) 


120 


IV.  The  purpose  of  this  part  of  the  questionnaire  is  to  determine  how  much 
the  mechanic  has  improved  in  his  knowledge  of  how  engines  work  and 
what  can  go  wrong  with  them  since  his  assignment  to  your  unit.  The 
kinds  of  knowledge  listed  below  are  the  same  as  those  in  Part  III. 
above.  This  time,  however,  you  are  to  describe  how  the  mechanic  has 
changed  in  his  knowledge  about  engines  since  he  was  assigned  to  your 
unit.  For  each  kind  of  knowledge,  please  circle  the  number  on  the 
right  that  best  describes  his  improvement.  If  the  mechanic  needed 
little  or  no  improvement  in  knowledge  when  he  was  first  assigned  to 
your  unit,  that  is,  if  he  knew  about  as  much  as  he  needs  to  know  for 
his  job,  please  place  an  X  in  the  column  headed  "No  improvement 
needed."  Also,  circle  the  number  for  "Knows  less"  or  "Equally  able" 
for  that  kind  of  knowledge  to  indicate  whether  the  mechanic  has  retained 
this  knowledge.  The  other  column  headings  are  explained  below. 

1.  Knows  less:  means  that  the  mechanic  has  forgotten  more  than  he 
has  learned  about  this  ki.^d  of  knowledge  since  he  was  first 
assigned  to  this  unit. 

2.  No  change:  means  that  the  mechanic  knows  as  much  (no  more,  no 
less)  about  this  subject  now  as  when  he  was  first  assigned  to 
this  unit. 

3.  Some  improvement :  means  that  the  mechanic  now  knows  somewhat 
more  about  this  subject  than  he  did  when  he  was  assigned  to 
this  unit. 

Considerable  improvement;  means  that  the  mechanic  now  knows  a 
great  deal  more  about  this  subject  than  be  did  when  he  was  first 
assigned  to  this  unit. 

Insufficient  information:  means  that  you  have  not  had  enough 
opportunity  to  observe  the  mechanic  to  evaluate  his  improvement 
in  knowledge  of  this  subject.  If  this  is  the  case  for  a  given 
subject,  place  an  X  in  this  column  opposite  that  subject. 


Kind  of  knowledge 


% 

0) 

»H 

u 

0) 

CO 

c 

a 

> 

(0 

0) 

V 

(d 

0) 

o 

00 

Q 

u 

e 

u 

'U 

c 

i' 

0) 

(U 

s* 

n) 

t> 

> 

cn 

o 

1-1 

o 

•H 

'V 

» 

o 

0!  W 

CO 

o 

1) 

o 

c 

o 

i  i* 

c 

o 

c 

:2 

2; 

CA  tH 

u 

(Kinds  of  knowledge  will  be  1234 

the  same  as  in  Part  III.) 

12  3  4 
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2 


3 
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Insufficient 

information 


V.  The  purpose  of  this  part  of  the  questionnaire  is  to  give  you 
an  opportunity  to  express  your  feelings  about  this  me:hanic'; 

AIT  training  in  general.  Please  answer  each  question  as  it 
applies  to  this  mechanic. 

Do  you  feel  that  this  mechanic  can  do  the  tasks  liste  1  in 
Part  1.  above  as  well  as  he  can  be  expected  to  do  the  ^  at 
this  point  in  his  career?  (Check  one.) 

_ Yes 

No 


Uncertain 


If  your  answer  was  No  or  Unc(  -ain,  which  tasks  do  yo  feel  he 
was  not  prepared  for  as  well  j  he  should  have  been?  (Please 
write  your  answer.) 


What  changes  should  be  made  in  the  AIT  training  progr.  m  to  help 
it  provide  the  training  necessary  at  this  stage?  (PI. ce  an  X 
in  the  blank  beside  each  change  you  feel  should  be  matie.  If 
you  are  not  familiar  with  some  aspects  of  AIT  traininj’  covered 
below,  simply  leave  the  blank  unmarked.) 


No  changes  needed 


More  time  is  needed  for  AIT  training 


Less  time  is  needed  for  AIT  training 


More  equipment  hands-on  training  is  needed 


The  AIT  training  equipment  should  be  improved 


_The  AIT  training  equipment  should  be  used  more 


More  theory  of  how  engines  operate  should  be  covered 


Less  theory  of  how  engines  operate  should  be  covered 


More  AIT  training  time  should  be  spent  on  hew  to  do 


maint  enance  tasks 


Less  AIT  training  time  should  be  spent  on  how  to  d(' 
maint  enance  tasks 


The  AIT  training  should  be  different  (different  in  what 
way? _ _ _ 


AIT  training  should  go  deeper  into  some  topics  (which 
topics?) _ _ _ 


Less  AIT  training  time  should  be  spent  on  some  topics 
(which  topics?) _ 


In  terms  of  the  capabilities  this  mechanic  should  have  at  this  stage 
in  his  career,  »»hat  do  you  feel  he  is  especially  well  prepared  to 
do? 


What  is  h'j  especially  weak  in? 


VI.  All  in  all,  how  valuable  was  the  training  equipment  used  for 
teaching  this  mechanic  about  engines  in  AIT?  (C^ieck  one  blank.) 

_ Very  valuable 


Valuable,  but  it  could  have  been  better 


Of  some  value,  but  a  lot  of  improvement  is  needed 


Of  no  value 


Uncertain 


All  in  all,  how  valuable  was  this  mechanic's  AIT  training  on  engines? 
(Check  one  blank.) 

_ Very  valuable 


Valuable,  but  it  could  have  been  better 


Of  some  value,  but  a  lot  of  improvement  is  needed 


Of  no  value 


Uncertain 
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Regardless  of  AIT  training,  what  is  your  overall  impression  of 
this  mechanic's  present  ability  to  maintain  and  repair  engines? 
(Circle  one.) 

Very  well  qualified 

Adequately  qualified 


_ Barely  qualified 

_ Not  qualified 

Whether  or  not  the  mechanic  is  realistic,  how  much  confidence  does 
he  have  in  his  ajili*'y  to  maintain  and  repair  engines?  (Check  one.) 

_ Feels  sure  he  can  do  a  good  job  on  almost  any  task 

_ Feels  he  can  do  a  good  job  most  of  the  time 

_  Feels  he  cannot  do  a  good  job  most  of  the  time 

_ ^Feels  that  he  is  not  able  to  do  the  job  recuired  of  him 

Are  the  mechanic's  feelings  about  his  ability  realistic?  That  is, 
does  he  show  overconfidence,  too  little  confidence,  or  just  the 
right  amount?  (Check  one.) 

_  Shows  overconfidence;  he  is  not  as  good  as  he  thinks  he  is 

_ ^Shows  too  little  confidence;  he  is  better  tnan  he  thinks  he 

_ He  is  realistic  in  his  feelings  about  his  ability 

How  does  this  mechanic  feel  about  his  job?  (Check  one.) 

_ ^Seems  very  enthusiastic;  seems  to  enjoy  his  job  ve  y  much 

_ ^Seems  happy  enough  with  his  job,  but  sometines  seems  not 

to  enjoy  it 

_ ^Seems  discontented  with  his  job,  but  makes  the  best  of  it 

_ ^Seems  very  dissatisfied  with  his  job;  he  sheuld  be  doing 

something  else 

Please  use  the  space  below,  or  backs  of  pages  if  needjd,  to  write 
comments  that  will  help  in  finding  out  how  valuable  ALT  training 
on  engines  is.  You  can  also  use  this  space  if  you  wish  to 
comment  on  the  questions  you  have  answered. 


i/This 


information  is  needed  only  once  from  each  sun^-rvisor. 
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